Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable cross validation support #38

Merged
merged 6 commits into from
Dec 6, 2021
Merged

Enable cross validation support #38

merged 6 commits into from
Dec 6, 2021

Conversation

wiseodd
Copy link
Collaborator

@wiseodd wiseodd commented Aug 26, 2021

@runame please review

@wiseodd wiseodd requested a review from runame August 26, 2021 12:28
Copy link
Collaborator

@runame runame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change to support regression looks good. There might be loss functions which also require the output variance, but for now the mean is enough.

Regarding the example script, I think it is very similar to the existing regression example, so maybe we can create a new directory examples and collect examples which we don't explicitly explain in the docs there. We could also do this on a separate branch for now. What do you think?

@wiseodd
Copy link
Collaborator Author

wiseodd commented Aug 30, 2021

Updated!

  • utils.py always return variance whenever available
  • optimize_prior_precision has an optional argument loss_with_var to handle your concern
  • added example directory

@runame
Copy link
Collaborator

runame commented Dec 6, 2021

@wiseodd Is it ok if I remove the examples/regression_example_mc.py file? If I remember correctly we decided to only have examples in this repository which will also be shown in the docs, i.e. the original regression one, your new calibration example, and then the continual learning one.

@runame runame added this to the NeurIPS Prerelease milestone Dec 6, 2021
@wiseodd
Copy link
Collaborator Author

wiseodd commented Dec 6, 2021

Yes, please go ahead

@runame runame merged commit cc81c44 into main Dec 6, 2021
@runame runame deleted the cross_validation branch December 6, 2021 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants