-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compare exp and softplus transform using synthetic data (cell2location model) #14
Conversation
This reverts commit 3b8e14e.
The issue with pyro was overlapping plate dimension - now replaced with expand/to_event (excluding obs_axis). The analysis essentially confirms the same observation, only setting 3 using softplus for all positive transformations retains original accuracy (see 2D histogram below for pymc3 and the notebook for pyro and numpyro):
@fritzo what do you think? Please let me know if you have any question and if I need to give more explanations in the notebook. This model can be described as a GLM/non-negative factor analysis where factor loadings for variables are provided as fixed and the goal is to learn loadings for observations. I can also do the analysis of ELBO stability using a model where factor loadings for variables also need to be learned. I expect this trend to be stronger because the model is less constrained - but I do not have the same ground truth for evaluating accuracy. |
@vitkl Thanks for setting the experiments! This is so cool. I would like to walk through this to understand better the behavior. I am a bit busy this week so will get back to you sometime early next week. :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @vitkl, these experiments look great! @fehiepsi and I discussed and agreed that:
- Your experiments are sufficient evidence to change autoguide scale parameters to use softplus transforms by default.
- While it would be too big a change to alter the default transform for
constraints.positive
, we would be happy to add documentation on how change the default, e.g. adding to the new Tips & Tricks tutorial.
@vitkl is this ready to merge? |
Let me quickly check that the text in the notebooks makes sense.
…On Sun, 25 Apr 2021, 19:09 Fritz Obermeyer, ***@***.***> wrote:
@vitkl <https://github.com/vitkl> is this ready to merge?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#14 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFMFTV4OXCOWO2W75D2U63TTKRLGVANCNFSM42Y3JD7Q>
.
|
@vitkl mind if I merge this? |
Thanks for your patience! I just cleaned up the notebooks a bit. I think you can merge this. |
Adding the experiments which compare the effect of exp and softplus transform on the cell2location model estimation (https://github.com/BayraktarLab/cell2location). Stability of ELBO and accuracy using ground truth estimates (R^2, PR curves) are compared on synthetic data. The cell2location model is ported from pymc3 to both numpyro and pyro:
The model is slightly different to the original pymc3 implementation:
Gamma(mu, sigma)
had to be re-parameterised toGamma(alpha, beta)
because PyTorch and numpyro do not supportGamma(mu, sigma)
.NB(mu, alpha also called theta and total count)
, pyro usesNB(logits=log(mu) - log(alpha), total count)
and numpyro usesGammaPoisson(alpha, beta=alpha / mu)
.Three conditions are compared:
For numpyro, the results are as follows:
For pyro, for some reason, this implementation gives NaN in all three comparisons after just a few iterations. Any thoughts about potential solutions would be appreciated. Maybe I am using the plates interface incorrectly?