Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only initialize runner once in NeymanConstructor #103

Closed
wants to merge 4 commits into from
Closed

Conversation

dachengx
Copy link
Collaborator

@dachengx dachengx commented Nov 2, 2023

This is a further fix of #100. We do not need to initialize runner for each **{**nominal_values, **generate_values}, but we can pass generate_values, nominal_values to runner.update_poi to pass correct nominal_values to get the expectations. Additionally, I made modifications:

  1. Require parameter's ptype to be in ["rate", "shape", "efficiency", "livetime"], the default ptype is "shape".
  2. Recover StatisticalModel.set_nominal_values and Parameters.set_nominal_values, add option to modify the static parameter so that user is aware of the modification.

@dachengx dachengx requested a review from hammannr November 2, 2023 19:00
Copy link

github-actions bot commented Nov 2, 2023

Pull Request Test Coverage Report for Build 6765927051

  • 11 of 12 (91.67%) changed or added relevant lines in 3 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.1%) to 91.57%

Changes Missing Coverage Covered Lines Changed/Added Lines %
alea/parameters.py 5 6 83.33%
Files with Coverage Reduction New Missed Lines %
alea/model.py 2 85.56%
Totals Coverage Status
Change from base Build 6745147134: -0.1%
Covered Lines: 1347
Relevant Lines: 1471

💛 - Coveralls

# then the poi_expectation is not in the nominal_expectation_values
component = self.poi.replace("_rate_multiplier", "")
poi_expectation = expectation_values.get(component, None)
if runner.input_poi_expectation:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just "if poi_expectation" in generate_values?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should also be fine. What is the benefit?

Comment on lines +198 to +199
generate_values = runner.update_poi(
runner.model, self.poi, generate_values, nominal_values
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this won't work with static parameters. That was the entire point of the previous PR

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran it quickly and yes, this crashes if you have static parameters (as it should).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The recent commit 7b4f38f makes the PR compatible with static parameters.

fittable: false
blueice_anchors:
- 50
description: WIMP mass in GeV/c^2
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This modification is to be compatible with the requirement of ptype.


def _define_parameters(self, parameter_definition, nominal_values=None):
def _define_parameters(self, parameter_definition):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think _define_parameters should be as simple as just initializing Parameters.

@@ -239,6 +236,18 @@ def store_data(
raise ValueError("The number of data sets and data names must be the same")
toydata_to_file(file_name, _data_list, data_name_list, **kw)

def set_nominal_values(self, overwrite_static=False, **nominal_values):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recover set_nominal_values, with an option to overwrite static parameters. And in principle I think we do need a method to overwrite static parameters(bypass the check).

@@ -54,6 +54,11 @@ def __init__(
self.name = name
self._nominal_value = nominal_value
self.fittable = fittable
if ptype not in ["rate", "shape", "efficiency", "livetime"]:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here the requirement of ptype is more strict.

@@ -226,7 +238,7 @@ def submit(
"threshold": [],
"poi_expectation": [],
}
threshold[threshold_key] = threshold_value
threshold[threshold_key] = deepcopy(threshold_value)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very important because the shallow copy will convey the modification of nominal_values and generate_values to threshold_value and threshold.

@@ -449,7 +449,7 @@ def convert_variations(variations: dict, iteration) -> list:
if not isinstance(v, list):
raise ValueError(f"variations {k} must be a list, not {v} with {type(v)}")
variations[k] = expand_grid_dict(v)
result = [dict(zip(variations, t)) for t in iteration(*variations.values())]
result = [dict(zip(variations, deepcopy(t))) for t in iteration(*variations.values())]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very important because the shallow copy will convey the modification of items in t in one entry of result, to another entry of result.

@dachengx dachengx requested review from kdund and hammannr November 6, 2023 01:16
@dachengx dachengx changed the title Only initial runner once in NeymanConstructor Only initialize runner once in NeymanConstructor Nov 6, 2023

self._check_ll_and_generate_data_signature()
self.set_nominal_values(overwrite_static=True, **kwargs.get("nominal_values", {}))
Copy link
Collaborator Author

@dachengx dachengx Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hammannr Can we keep set_nominal_values in the future so that _define_parameters only handle the definition of Parameters? And for the need_reinit parameters, I think they can be defined correctly just by set_nominal_values.

@hammannr
Copy link
Collaborator

This PR can be closed now, right @dachengx ?

@dachengx
Copy link
Collaborator Author

Suspended by #108

@dachengx dachengx closed this Nov 13, 2023
@dachengx dachengx deleted the fast_neyman branch January 19, 2024 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants