Customize the number of samples taken by DynamicHMC

[DilumAluthge]

It would be nice if the user could specify e.g. "take 100_000 samples (after the warmup) from NUTS`, and have this value passed to DynamicHMC.

This corresponds to N argument:

N: the number of samples for inference, after the warmup.

From here: https://tamaspapp.eu/DynamicHMC.jl/latest/interface/#DynamicHMC.mcmc_with_warmup

[cscherrer]

We really need a uniform sampling interface in Soss, may be good to do something similar to Turing's. In Soss, dynamicHMC calls this:

function dynamicHMC(
    rng::AbstractRNG,
    m::JointDistribution,
    _data,
    N::Int = 1000;
    method = logpdf,
    ad_backend = Val(:ForwardDiff),
    reporter = DynamicHMC.NoProgressReport(),
    kwargs...,
)

So it's there, but some things could be wired together more cleanly

[DilumAluthge]

So in the linear regression example, if I want to set N = 1234, what would I change?

[cscherrer]

Do we want it as a field for the model?

[cscherrer]

e.g. this works:

julia> m = @model N begin
           p ~ Uniform()
           y ~ Binomial(N,p)
       end;

julia> dynamicHMC(m(N=100), (y=20,), 5)
5-element Array{NamedTuple{(:p,),Tuple{Float64}},1}:
 (p = 0.18743530744069434,)
 (p = 0.1786246661840475,)
 (p = 0.16361538929852976,)
 (p = 0.1825751595928423,)
 (p = 0.17100162031893165,)

[cscherrer]

so maybe in the model specification we have numSamples, and then pass that along in this way when the sampler is called?

[DilumAluthge]

I think so.

For reference, maybe we can look at what https://github.com/alan-turing-institute/MLJFlux.jl does.

Where do they store their "num_epochs" or "num_iterations" or whatever?

[cscherrer]

Where do they store their "num_epochs" or "num_iterations" or whatever?

hmm, not sure. I'd guess in this context you'd generally want early stopping. Hhaven't really seen that used in Bayesian modeling though, don't see it applying in quite the same way

[DilumAluthge]

FWIW it looks like they also store it as a field in their model, which is the same thing you suggested.

[DilumAluthge]

Also, I'd like to be able to customize the value passed to the reporter keyword argument.

[DilumAluthge]

Also, I'd like to be able to customize the value passed to the reporter keyword argument.

Actually, this part should be controlled by the verbosity argument of fit. I.e. if verbosity = 0, then DynamicHMC logging should be disabled. But if verbosity = 1, then DynamicHMC should print (via @info) the information about the progress of the sampler.

[cscherrer]

There's some subtlety here, because we need this to be flexible for use with lots of different inference methods. @DilumAluthge what do you think of SossMLJModel having fields hyperparams, transform, model, infer, and kwargs

Then for inference, we call

infer(model(merge(transform(X), hyperparams)), (y=y,); kwargs...)

[DilumAluthge]

Sounds great!

[DilumAluthge]

Well it's not just keyword arguments. `N' is a positional argument.

So it is both keyword arguments and positional arguments. Maybe inference_parameters?