1.6.0

Breaking changes

• Update to PyTorch 1.8 release (required).
• Enable validation by default #2701. To disable globally call pyro.enable_validation(False); or disable locally to one distribution via e.g. Normal(loc, scale, validate_args=False).
• Switch from LKJCorrCholesky distribution to upstream LKJCholesky distribution #2771.

New Tutorials

• Product LDA for probabilistic topic modeling #2729.

New features

• Support masked conditioning via pyro.sample(..., obs_mask=...) #2772.
• Distributions:
  • Added a Distribution.infer_shapes() method for static shape analysis #2739.
  • Added ProjectedNormal for circular and spherical data (with more functionality than VonMises) #2736.
  • Added AffineBeta distribution over arbitrary intervals #2735 by @ordabayevy.
  • Added SpanningTree properties .mode, .edge_mean #2727
  • Added OneOneMatching and OneTwoMatching distributions to approximately marginalize over bipartite matchings #2707, #2697.
• Reparametrizers:
  • Added ProjectedNormalReparam for inference with ProjectedNormal distributions #2736.
  • Support more batch distributions in HaarReparam #2731.
• Constraints:
  • Added a Constraint.event_dim property #2753.
  • Added positive_ordered_vector, corr_matrix #2762
  • Added sphere #2736.
  • Added softplus_positive and softplus_lower_cholesky constraints with numerically stable SoftplusTransform and SoftplusLowerCholeskyTransform #2767.
• Added a TraceMarkovEnum_ELBO and VectorizedMarkovMessenger for parallel scan enumeration #2703, #2703 by @ordabayevy.
• Added an experimental autoname handler #2745 by @ordabayevy.
• Added goodness of fit tests for testing new distribution implementations #2738.
• Support MaskedDistribution in contrib.forecast #2709.

Bugfixes

• Many fixes to distribution shapes #2759, #2746, #2739.
• Detach AutoGuide.median() and .quantiles() methods #2743.

1.5.2

This patch release merely

• Pins to requirements to torch<1.8 to avoid breaking changes in torch 1.8.0 (introduced in pytorch/pytorch#50547 pytorch/pytorch#50581).
• Fixes an fft bug #2731

1.5.1

New features

• #2693 A GumbelSoftmaxReparam for relaxed categorical distributions

Bug fixes

• #2683 Support PyTorch 1.7
• #2682 Fix help(MyDistribution)
• #2679 Fix TraceEnum_ELBO.compute_marginals()
• #2677 Warn if infer_discrete() finds no discrete sites

1.5.0

New features

• New Tutorials:
  • Deep Generative Modeling with Single Cell Data
  • Conditional Variational Autoencoder
  • Forecasting with Dynamic Linear Models
• New distributions and transforms:
  • OrderedLogistic distribution and OrderedTransform
  • ConditionalMatrixExponential normalizing flow
  • ConditionalSplineAutoregressive normalizing flow
• contrib.forecast now supports HaarReparam, DiscreteCosineReparam, and poutine.trace to record posterior samples of latent variables.
• CompartmentalModel now supports a .finalize() method to add likelihoods that couple states across time.
• Integration with Funsor, an experimental intermediate language for probabilistic programming
  • pyro.contrib.funsor is a new backend for Pyro that aims to simplify the implementations of Pyro's most powerful inference engines. For details, see: tutorial 1, tutorial 2, example usage with pyroapi
  • poutine.collapse and pyro.barrier provide experimental support for collapsing conjugate fragments within existing inference algorithms, using Funsor under the hood.

Breaking changes

• Require PyTorch 1.6.
• Drop support for Python 3.5; require Python 3.6+.
• Zero inflated distributions changed interface. #2643

Bug fixes & performance tweaks

• pyro.factor statements are now allowed in guides without warning. #2664
• Fix model-directed subsampling in autoguides. #2638
• Fix sample shape bug in LKJCorrCholesky distribution. #2617
• Speed up log-matmul-exp operations in discrete enumeration and DiscreteHMM. #2640
• Fix potential_fn issues in MCMC. #2591

1.4.0

New features

• A new pyro.contrib.epidemiology module for discrete-state discrete-time stochastic compartmental models. #2426
• New tutorials on:
  • Normalizing flows
  • Epidemiology
  • PyroModule
  • MAP/MLE inference
• New transforms and normalizing flows:
  • Monotonic rational quadratic Spline flow.
  • A SplineCoupling flow.
  • A SplineAutogregressive flow.
  • A MatrixExponential flow.
  • Conditional versions of AffineAutoregressive, Householder, NeuralAutogregressive, Spline, and GeneralizedChannelPermute flows.
  • A Haar wavelet transform and reparameterizer.
  • Permute and AffineCoupling can operate on a specific dimension with dim keyword argument #2472
• New distributions:
  • CoalescentTimes, CoalescentTimesWithRate, and CoalescentRateLikelihood for coalescent processes in phylogenetics.
  • TruncatedPolyGamma.
  • ExtendedBinomial and ExtendedBetaBinomial with relaxed support.
  • ImproperUniform for expressing factor graphs.
• Improvements to MCMC:
  • structured mass matrix adaptation #2425 #2473
  • arrowhead precision matrix adaptation #2465
  • support for randomized init strategies like init_to_generated #2417
• Improvements to SMC:
  • systematic resampling #2488
  • resampling based on effective sample size (ESS) #2486
• A new SplitReparam allows different inference strategies to apply to different parts of a tensor random variable.
• A new initialization strategy init_to_generated.
• A RandomVariable container class to support method chaining syntax for transforming distributions #2448

Bug fixes

• Support sequential plates in RenyiELBO #2541
• Fixes to AffineAutogregressive #2504
• Fixes to BatchNorm TransformModule #2459
• Fixes to how some transforms handle parameters #2544
• Fixes to reraising logic that clean up error reporting during inference #2494
• many other fixes to documentation and code

1.3.1

New features

• A new Spline transform which implements element-wise rational spline bijections of linear order.
• A new ConditionalAffineCoupling transform which implements the affine coupling layer of RealNVP that conditions on an additional context variable.

Enhancements to the pyro.contrib.forecast module

• Support drawing samples in batches.
• Add walltime to backtest to measure performance of model training and forecasting.
• Support more likelihood distributions: Geometric, NegativeBinomial, ZeroInflatedNegativeBinomial.

Bug fixes

• #2399 raises an error when HMC/NUTS is used for a model with subsampling.
• #2390 makes PyroModule compatible with torch.nn.RNN.
• #2388 allows unused params in CSIS inference.
• #2384 fixes some caching issues in calculation of log_abs_det_jacobian of TransformModules
• #2365 fixes a naming bug in LocScaleReparam whereby all loc-scale reparameterized sites shared a single centeredness parameter.
• #2355 makes jit_compile=True flag in HMC/NUTS work for models with pyro.param statements.

1.3.0

New features

• A new AutoNormal guide that supports data subsampling, thanks to @patrickeganfoley.
• Data subsampling support for the AutoDelta guide.
• A new pyro.subsample primitive to aide in subsampling.
• An AutoNormalizingFlow autoguide.
• A new pyro.contrib.forecast module for multivariate hierarchical heavy-tailed forecasting, together with three tutorials: uivariate, heavy-tailed, state space models, and hierarchical models.
• A tutorial on Boosting Black Box Variational Inference, thanks to @lorenzkuhn, @gideonite, @sharrison5, and @TNU-yaoy.
• A NeuTraReparam example.
• PyroModule's interface is now stable, no longer EXPERIMENTAL.
• Better validation for GaussianHMM and LinearHMM distributions.
• Miscellaneous new GaussianHMM method to handle conjugacy.

Bug fixes

• #2345 remove pillow-simd dependency
• #2327 Make pyro.deterministic not warn when called outside of inference
• #2321 Support plates in RenyiELBO
• #2266 Fixes to transform handling in MCMC api

1.2.1

Patches 1.2.0 with the following bug fixes:

• Fix for MCMC with parallel chains using multiprocessing, where transforms to the latent sites' support was not being correctly stored.
• Other minor rendering related fixes for tutorials.

1.2.0

Misc changes

• Updated to PyTorch 1.4.0 and torchvision 0.5.0.
• Changed license from MIT to Apache 2.0 and removed Uber CLA as part of Pyro's move to the Linux foundation.

Reparameterization

This release adds a new effect handler and a collection of strategies that reparameterize models to improve geometry. These tools are largely orthogonal to other inference tools in Pyro, and can be used with SVI, MCMC, and other inference algorithms.

• poutine.reparam() is a new effect handler that transforms models into other models for which inference may be easier (Gorinova et al. 2019).
• pyro.infer.reparam is a collection of reparameterization strategies following a standard Reparam interface:
  • Decentering transforms for location-scale families (Gorinova et al. 2019).
  • Transform unwrapping to deconstruct TransformedDistributions.
  • Discrete Cosine transforms for frequency-domain parameterizations (useful for inference in time series).
  • Auxiliary variable methods for Levy Stable and StudentT distributions.
  • Linear Hidden Markov Model reparameterization, allowing a range of non-Gaussian HMMs to be treated as conditional Gaussian processes.
  • Neural Transport uses SVI to learn the geometry of a model before drawing samples using HMC (Hoffman et al. 2019).
• A tutorial on inference with Levy Stable distrubutions, demonstrating StableReparam, DiscreteCosineReparam, and EnergyDistance.

Other new features

• A tutorial on Dirichlet process mixture modeling, contributed by @m-k-S
• Added a LinearHMM distribution with an .rsample() method. This supports non-Gaussian noise such as Levy Stable and StudentT, but requires reparameterization for inference.
• Implemented a GaussianHMM.rsample() method for drawing joint samples from a linear-Gaussian HMM.
• Added a LowerCholeskyAffine transform.
• #2264 improves speed and numerical stability of MultivariateNormal conversion from scale_tril to precision.

Bug fixes

• #2263 fixes MCMC api to allow implementations other than HMC and NUTS.
• #2244 fixes an event_dim issue in ConditionedPlanar flow.
• #2243 fixes a bug in AffineCoupling.
• #2227 fixes device placement of the MultivariateStudentT.df param.
• #2226 fixes an edge case bug in discrete enumeration.

1.1.0

New Features

• pyro.infer.ReweightedWakeSleep implements the Reweighted Wake Sleep algorithm (Le et al. 2019). Contributed by Siddharth Narayanaswamy and Tuan Anh Le.

• pyro.infer.TraceTMC_ELBO implements the Tensor Monte Carlo marginal likelihood estimator (Aitchinson 2019), a generalization of the importance-weighted autoencoder objective.

• pyro.infer.EnergyDistance implements a likelihood-free inference algorithm based on Szekely's energy statistics, a multidimensional generalization of CRPS (Gneiting & Raftery 2007).

• pyro.contrib.cevae implements the Causal Inference VAE of (Louizos et al. 2017). See examples/contrib/cevae/synthetic.py for an end-to-end usage example.

• pyro.deterministic primitive to record deterministic values in the trace.

• pyro.nn.to_pyro_module_() recursively converts an regular nn.Module to a PyroModule in-place.

• A default implementation for Distribution.expand() that is available to all Pyro distributions that subclass from TorchDistribution, making it easier to create custom distributions.

New distributions and transforms

• MultivariateStudentT is a heavy-tailed multivariate distribution.
• Stable implements a Lévy α-stable distribution with reparametrized .rsample() method but no .log_prob(). This can be fit using EnergyDistance inference.
• ZeroInflatedNegativeBinomial is a distribution for count data.
• LowerCholeskyAffine is a multivariate affine transform.

Other Changes / Bug Fixes

• pyro.util.save_visualization has been deprecated, and dependency on graphviz is removed.
• #2197 fixed a naming bug in PyroModule that affected mutliple sub-PyroModules with conflicting names.
• #2192 Bug fix in Planar normalizing flow implementation
• #2188 Make error messages for incorrect arguments to effect handlers more informative