Clean up exports #2474

penelopeysm · 2025-01-21T00:07:38Z

Closes #2481
Closes #2468

As mentioned in #2468, I don't think we need to export so many things. I've done a bunch of cuts here, with the intention that if we don't feel comfortable cutting something, we can easily revert specific commits. In my opinion, though, even though it looks like many sweeping changes, I don't think any of these are hugely controversial on their own.

Concretely, this PR does the following:

Stops re-exporting the DynamicPPL module.
Stops re-exporting @logprob_str and @prob_str, which were deprecated several months ago (we've killed off deprecated stuff in far shorter timespans)
~~Stops re-exporting DynamicPPL.LogDensityFunction, this is an implementation detail that users probably don't need to construct themselves.~~
Bumps the Distributions.jl compat bound (from 0.23 to 0.25.77) and removes code that was required to support old versions of Distributions. Note that DynamicPPL already requires Distributions 0.25.
Removes the blanket re-export of Bijectors and Libtask, as well as the export of Bijectors.ordered. ordered is used in the Gaussian mixture model tutorial, but even there it's imported directly from Bijectors, and I don't think it's unfair to expect users to manually import Bijectors if they want to use this function.
Removes the blanket re-export of AbstractMCMC, replacing it with an explicit export list of sample, MCMCThreads, MCMCDistributed, and MCMCSerial. Note that these are the only four identifiers which AbstractMCMC itself exports (https://github.com/TuringLang/AbstractMCMC.jl/blob/master/src/AbstractMCMC.jl) so the only change is that the module name itself (AbstractMCMC) is no longer accessible as Turing.AbstractMCMC.
For convenience, re-exports LinearAlgebra.I.
Exports DynamicPPL.returned and DynamicPPL.prefix (we forgot to re-export this when doing the submodel stuff).

Will let CI run before requesting a review.

codecov · 2025-01-21T00:22:33Z

Codecov Report

Attention: Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.

Project coverage is 86.03%. Comparing base (e842e30) to head (1cf5270).
Report is 3 commits behind head on breaking.

Files with missing lines	Patch %	Lines
src/mcmc/Inference.jl	0.00%	1 Missing ⚠️
src/stdlib/distributions.jl	75.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           breaking    #2474      +/-   ##
============================================
+ Coverage     85.53%   86.03%   +0.49%     
============================================
  Files            21       21              
  Lines          1466     1454      -12     
============================================
- Hits           1254     1251       -3     
+ Misses          212      203       -9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

coveralls · 2025-01-21T00:42:02Z

Pull Request Test Coverage Report for Build 12885951319

Details

0 of 6 (0.0%) changed or added relevant lines in 3 files are covered.
430 unchanged lines in 11 files lost coverage.
Overall coverage decreased (-27.7%) to 48.809%

Changes Missing Coverage	Changed/Added Lines	%
src/mcmc/Inference.jl	1	0.0%
src/optimisation/Optimisation.jl	1	0.0%
src/stdlib/distributions.jl	4	0.0%

Files with Coverage Reduction	New Missed Lines	%
src/mcmc/emcee.jl	2	88.46%
src/variational/VariationalInference.jl	4	0.0%
src/mcmc/particle_mcmc.jl	10	80.13%
src/mcmc/Inference.jl	20	75.15%
ext/TuringDynamicHMCExt.jl	29	0.0%
src/mcmc/mh.jl	34	58.96%
src/stdlib/distributions.jl	47	0.0%
ext/TuringOptimExt.jl	50	0.0%
src/mcmc/sghmc.jl	50	20.29%
src/variational/advi.jl	56	0.0%

Totals
Change from base Build 12786420948:	-27.7%
Covered Lines:	758
Relevant Lines:	1553

💛 - Coveralls

mhauru · 2025-01-21T09:56:52Z

I think I agree with all the changes listed in OP, although LogDensityFunction makes me pause and wonder how people might be using it. I don't think I should be the main reviewer here, the more senior members of the team understand user needs better.

Rather than merging straight to master, should we bundle this with #2458 to form a new breaking release to go out when we feel like is the right moment?

penelopeysm · 2025-01-21T11:04:28Z

Happy to release both PRs together!

I can't say I feel particularly bad about LogDensityFunction tbh, it's still accessible via DynamicPPL with the same functionality, so we're really just asking people to import it from where it should be imported.

penelopeysm · 2025-01-21T11:05:00Z

There's still the wholesale export of MCMCChains which I don't dare to touch. MCMCChains exports a lot of stuff. https://github.com/TuringLang/MCMCChains.jl/blob/master/src/MCMCChains.jl

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

penelopeysm · 2025-03-17T15:32:42Z

I changed my mind on LogDensityFunction, it can stay exported. I suspect that over the course of AbstractMCMC refactoring it'll also start to play a larger role.
Nevertheless, I changed all the Turing.LogDensityFunction calls in the library to DynamicPPL.LogDensityFunction which feels a bit cleaner.

I think it's almost ready for review, will let CI run before requesting one though, as I'm not sure if there are still some imports I need to fix in the test suite.

Docs preview: https://turinglang.org/Turing.jl/previews/PR2474/api/

mhauru

Looks good. Could we get a HISTORY.md entry? Maybe merge the base branch first before writing it.

Other than that, I still need to build the docs to see how they look but I'm happy otherwise.

docs/src/api.md

mhauru · 2025-03-18T13:18:54Z

src/essential/Essential.jl

-    AutoMooncake,
-    @logprob_str,
-    @prob_str
+    @varname, AutoForwardDiff, AutoReverseDiff, AutoMooncake, @logprob_str, @prob_str


I know this the auto-formatter, but it's quite ugly. Any setting we should set to stop this from happening? It bothers me locally too.

tldr is no: domluna/JuliaFormatter.jl#358

We could try doing

export a export b export c

instead. Part of me might actually prefer this (even though the keyword export is spammed) because it's more resistant towards moving lines up/down (you can't have a trailing comma at the end of the export list, so

export a, b, c,

doesn't work)

I don't mind spamming export either, or having a few per line that relate to each other if they fit on one line.

I changed it to

export @model export @varname export AutoForwardDiff, AutoReverseDiff, AutoMooncake

and JuliaFormatter seems happy ☺️

(Also removed that stray logprob_str and prob_str, oops.)

Co-authored-by: Markus Hauru <[email protected]>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

mhauru

Built docs look good except for one link.

docs/src/api.md

mhauru

Some proposed changes to HISTORY.md, feel free to reject or edit.

HISTORY.md

Co-authored-by: Markus Hauru <[email protected]>

mhauru

Thanks Penny!

* Bump minor version to v0.37 * Remove selector/space stuff (#2458) * Remove selector stuff from ESS * Remove selector stuff from MH * Remove selector stuff from HMC * Remove selector stuff from Emcee * Remove selector stuff from IS * Add missing getspace methods * Remove selector stuff for particle methods * Fix an HMC selector bug * Code style * Fix Emcee selector bug * Fix typo in ESS tests * Fix some constructor overwrites * Remove unnecessary tests * Remove selector stuff from SGHMC * Remove drop_space and other non-longer-necessary deprecation measures * Bump minor version 0.37. Add a HISTORY.md entry * Apply suggestions from code review Co-authored-by: Penelope Yong <[email protected]> * Remove unnecessary type parameters Co-authored-by: Penelope Yong <[email protected]> * Simplify constructors in particle_mcmc.jl * Remove calls to setgid and updategid --------- Co-authored-by: Penelope Yong <[email protected]> * Bump Mooncake compat to 0.4.95 * Support for DynamicPPL v0.35 (#2488) * Progress towards compat with DPPL v0.35 * More fixing of DPPL v0.35 stuff * Fix LogDensityFunction argument order * More minor bugfixes * [TEMP] Commit Manifest pointing to DynamicPPL#release-0.35 * remove LogDensityProblemsAD (#2490) * Remove LogDensityProblemsAD, part 1 * update Optimisation code to not use LogDensityProblemsAD * Fix field name change * Don't put chunksize=0 * Remove LogDensityProblemsAD dep * Improve OptimLogDensity docstring * Remove unneeded model argument to _optimize * Fix more tests * Remove essential/ad from the list of CI groups * Fix HMC function * more test fixes (#2491) * Remove LogDensityProblemsAD, part 1 * update Optimisation code to not use LogDensityProblemsAD * Fix field name change * Don't put chunksize=0 * Remove LogDensityProblemsAD dep * Improve OptimLogDensity docstring * Remove unneeded model argument to _optimize * Fix more tests * Remove essential/ad from the list of CI groups * Fix HMC function * More test fixes * Remove Manifest * More fixes for DynamicPPL 0.35 (#2494) * Remove test/dynamicppl/compiler.jl * Remove old regression tests * Remove vdemo2 * Fix last test * Add HISTORY.md entry about DPPL 0.35 * Allow ESS to sample variables with different symbols * Update a TODO note --------- Co-authored-by: Penelope Yong <[email protected]> * Fix call to DynamicPPL.initialize_parameters!! * Remove `Zygote` (#2505) * Remove `Zygote`; fix #2504 * Update test/test_utils/ad_utils.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Add HISTORY.md entry about removing support for Zygote --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Markus Hauru <[email protected]> Co-authored-by: Penelope Yong <[email protected]> Co-authored-by: Markus Hauru <[email protected]> * Fix a Gibbs test * Clean up exports (#2474) * Regroup exports by package * Export DynamicPPL.returned and DynamicPPL.prefix * Stop exporting @logprob_str and @prob_str * Remove DynamicPPL module export * Remove DynamicPPL.LogDensityFunction re-export * Remove BernoulliLogit, drop support for Distributions < 0.25.77 * Stop blanket re-exporting Libtask and Bijectors * Manually specify AbstractMCMC exports * Format Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Remove Bijectors.ordered export * Re-export LinearAlgebra.I * Replace Turing.Model -> DynamicPPL.Model * Format * Keep exporting LogDensityFunction * Add note in docs * Align Turing exports with docs API page * Fix things like `predict` on docs API page * Fix a Gibbs test * Format * Fix missing Bijectors import * Update docs/src/api.md Co-authored-by: Markus Hauru <[email protected]> * Update docs/src/api.md Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Clean up broken docs links, remove unneeded deps * Format * Add changelog entry for exports * Clean up exports in essential/Essential * Apply suggestions from code review Co-authored-by: Markus Hauru <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Markus Hauru <[email protected]> Co-authored-by: Markus Hauru <[email protected]> --------- Co-authored-by: Penelope Yong <[email protected]> Co-authored-by: Hong Ge <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

penelopeysm force-pushed the py/imports branch from 3eec503 to 6a8d318 Compare January 21, 2025 00:12

penelopeysm force-pushed the py/imports branch from d71170e to ebf8bf3 Compare January 21, 2025 00:40

penelopeysm force-pushed the py/imports branch 6 times, most recently from abc1f9b to d4c3547 Compare January 21, 2025 02:39

penelopeysm force-pushed the py/imports branch 2 times, most recently from 8fd6095 to 2409aa1 Compare January 21, 2025 11:26

Regroup exports by package

e5d182f

penelopeysm force-pushed the py/imports branch from 2409aa1 to 6c9093f Compare February 14, 2025 17:24

penelopeysm changed the base branch from master to breaking February 14, 2025 17:24

penelopeysm and others added 11 commits February 15, 2025 12:10

Export DynamicPPL.returned and DynamicPPL.prefix

b4ad4ae

Stop exporting @logprob_str and @prob_str

df41231

Remove DynamicPPL module export

d914e21

Remove DynamicPPL.LogDensityFunction re-export

a8c8601

Remove BernoulliLogit, drop support for Distributions < 0.25.77

a86ac33

Stop blanket re-exporting Libtask and Bijectors

e6e5961

Manually specify AbstractMCMC exports

98e53de

Format

2d48540

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Remove Bijectors.ordered export

efdf364

Re-export LinearAlgebra.I

e3c9695

Replace Turing.Model -> DynamicPPL.Model

e9c7cb5

penelopeysm force-pushed the py/imports branch from 6c9093f to e9c7cb5 Compare February 15, 2025 12:10

penelopeysm added 2 commits March 17, 2025 15:01

Align Turing exports with docs API page

433c36d

Fix things like predict on docs API page

1d036a1

penelopeysm force-pushed the py/imports branch from 7592b66 to 1d036a1 Compare March 17, 2025 15:26

penelopeysm marked this pull request as ready for review March 17, 2025 15:27

TuringLang deleted a comment from github-actions bot Mar 17, 2025

mhauru and others added 3 commits March 17, 2025 21:33

Fix a Gibbs test

ea4f977

Format

2bbd1c3

Fix missing Bijectors import

b6ccbee

penelopeysm force-pushed the py/imports branch from 246cc43 to b6ccbee Compare March 17, 2025 21:33

mhauru requested changes Mar 18, 2025

View reviewed changes

penelopeysm and others added 2 commits March 18, 2025 13:33

Update docs/src/api.md

df854dd

Co-authored-by: Markus Hauru <[email protected]>

Update docs/src/api.md

8fac6e4

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

mhauru requested changes Mar 18, 2025

View reviewed changes

docs/src/api.md Outdated Show resolved Hide resolved

penelopeysm added 4 commits March 18, 2025 13:44

Clean up broken docs links, remove unneeded deps

171091d

Format

74fd3b9

Add changelog entry for exports

763f203

Clean up exports in essential/Essential

e119b66

penelopeysm force-pushed the py/imports branch from f1dc019 to e119b66 Compare March 18, 2025 15:33

penelopeysm requested a review from mhauru March 18, 2025 15:38

mhauru requested changes Mar 18, 2025

View reviewed changes

HISTORY.md Outdated Show resolved Hide resolved

HISTORY.md Outdated Show resolved Hide resolved

HISTORY.md Outdated Show resolved Hide resolved

HISTORY.md Outdated Show resolved Hide resolved

Apply suggestions from code review

f0dd06a

Co-authored-by: Markus Hauru <[email protected]>

mhauru approved these changes Mar 18, 2025

View reviewed changes

Merge branch 'breaking' into py/imports

1cf5270

penelopeysm merged commit 625a9c8 into breaking Mar 18, 2025
2 checks passed

penelopeysm deleted the py/imports branch March 18, 2025 17:33

This was referenced Mar 27, 2025

prefix should be exported #2481

Closed

Clean up exports #2468

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up exports #2474

Clean up exports #2474

penelopeysm commented Jan 21, 2025 •

edited

Loading

codecov bot commented Jan 21, 2025 •

edited

Loading

coveralls commented Jan 21, 2025 •

edited

Loading

mhauru commented Jan 21, 2025

penelopeysm commented Jan 21, 2025

penelopeysm commented Jan 21, 2025

penelopeysm commented Mar 17, 2025 •

edited

Loading

mhauru left a comment

mhauru Mar 18, 2025

penelopeysm Mar 18, 2025

mhauru Mar 18, 2025

penelopeysm Mar 18, 2025

mhauru left a comment

mhauru left a comment

mhauru left a comment

Clean up exports #2474

Clean up exports #2474

Conversation

penelopeysm commented Jan 21, 2025 • edited Loading

codecov bot commented Jan 21, 2025 • edited Loading

Codecov Report

coveralls commented Jan 21, 2025 • edited Loading

Pull Request Test Coverage Report for Build 12885951319

Details

💛 - Coveralls

mhauru commented Jan 21, 2025

penelopeysm commented Jan 21, 2025

penelopeysm commented Jan 21, 2025

penelopeysm commented Mar 17, 2025 • edited Loading

mhauru left a comment

Choose a reason for hiding this comment

mhauru Mar 18, 2025

Choose a reason for hiding this comment

penelopeysm Mar 18, 2025

Choose a reason for hiding this comment

mhauru Mar 18, 2025

Choose a reason for hiding this comment

penelopeysm Mar 18, 2025

Choose a reason for hiding this comment

mhauru left a comment

Choose a reason for hiding this comment

mhauru left a comment

Choose a reason for hiding this comment

mhauru left a comment

Choose a reason for hiding this comment

penelopeysm commented Jan 21, 2025 •

edited

Loading

codecov bot commented Jan 21, 2025 •

edited

Loading

coveralls commented Jan 21, 2025 •

edited

Loading

penelopeysm commented Mar 17, 2025 •

edited

Loading