Fixes to AD backend usage in externalsampler

[torfjelde]

I was playing around with an implementation of AutoMALA (see https://arxiv.org/abs/2310.16782 and Pigeons.jl) using AbstractMCMC.jl for a presentation I'm doing on Turing.jl-related stuff, and I ran into some issues with externalsampler.

Specifically, somewhere along the way, a LogDensityProblemsAD.ADgradient(...) of a DynamicPPL.LogDensityModel becomes just a DynamicPPL.LogDensityModel, i.e. we lose the ability to compute gradients.

We didn't catch this in our tests because we both AbstractHMC.jl and AbstractMH.jl default to using ForwardDiff if needed, and so they would use the correct adtype in the very first iteration and then subsequently switch to using their own defaults :upside_down_smiley:

[torfjelde]

@yebai @devmotion @sunxd3 this should be a quick merge

[sunxd3]

looks good to me

[torfjelde]

Had to disable a few tests for MH, which doesn't actually follow the LogDensityProblems.jl interfacr properly (calls the method using NamedTuple instead of Vector). Two things here: a) all the NamedTuple cases are already easily supported by the internal MH sampler, so we're not losing any features here, and b) this is something that shoud be addressed in AdvancedMH.jl, not here.

EDIT: Tests should be passing now though, so if people are happy with the above, then we're gucci.

[torfjelde]

Uhmm the test failures seem very strange o.O I cannot reproduce this on my own machine; it's not windows, but I can't see how we can get undef behaviors depending on the OS o.O

EDIT: I lied 🤦 Was using the incorrect branch locally..

[torfjelde]

It seems to be the same issue as JuliaFolds2/BangBang.jl#21 but we're using the correct version of BangBang.jl, indicating that the fix I did in JuliaFolds2/BangBang.jl#22 did not cover ReverseDiff.jl for some reason (though I'm also confused as to why we haven't caught this before).

[torfjelde]

Okay, so now this PR should be passing. Disabled the test as mentioned in TuringLang/DynamicPPL.jl#612.

Once those tests have passed, this should be good to go.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 0.00%. Comparing base (abb7981) to head (cbb1e59).
Report is 3 commits behind head on master.

Files	Patch %	Lines
src/mcmc/abstractmcmc.jl	0.00%	3 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #2223   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files          22      22           
  Lines        1528    1529    +1     
======================================
- Misses       1528    1529    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[coveralls]

Pull Request Test Coverage Report for Build 9140038894

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 3 (0.0%) changed or added relevant lines in 1 file are covered.
• 3 unchanged lines in 1 file lost coverage.
• Overall coverage remained the same at 0.0%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/mcmc/abstractmcmc.jl	0	3	0.0%

Files with Coverage Reduction	New Missed Lines	%
ext/TuringOptimExt.jl	3	0.0%

Totals
Change from base Build 8989896440:	0.0%
Covered Lines:	0
Relevant Lines:	1529

---

💛 - Coveralls