Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DAQ] Bootstrax dynamic adapt targets for AmBe high rate #1492

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

cfuselli
Copy link
Member

@cfuselli cfuselli commented Dec 3, 2024

We introduce this PR to allow a dynamic modification of targets processed on the DAQ during difficult times like AmBe calibrations.

We add a specific case for which if "ambe" is in the run mode name, the targets and post-targets are updated based on some configuration that is defined in the DAQ database, separately for each of the event builders. This PR is just to allow for a dynamic change of targets. The nice game of deciding who will process what at a given time is implemented in the scripts used on the DAQ and in daqnt (See related daqnt PR https://github.com/XENONnT/daqnt/pull/110).

This method allowed us to survive and successfully process the last two heavy rate AmBe campaigns.

On top of that, if "ambe" is in the mode name, the data rate used to calculate how many resources to use ( cores and workers ) is set to the max of 550MB/s, such that a low amount of resources will be chosen.

  • New argument fix_resources: it is just for testing, it just ignores the n_failures in deciding how many resources to use.

@cfuselli
Copy link
Member Author

cfuselli commented Dec 3, 2024

Goes together with https://github.com/XENONnT/daqnt/pull/110

@cfuselli cfuselli changed the title [DAQ] Bootstrax dynamic adapt resources for AmBe high rate [DAQ] Bootstrax dynamic adapt targets for AmBe high rate Dec 3, 2024
@cfuselli cfuselli marked this pull request as ready for review December 3, 2024 13:17
@cfuselli cfuselli requested review from yuema137 and mflierm December 3, 2024 13:18
@coveralls
Copy link

coveralls commented Dec 3, 2024

Coverage Status

coverage: 89.501% (+0.02%) from 89.48%
when pulling 2987dcc on ambe_from_db_v3.0.0
into f765c10 on master.

Copy link
Collaborator

@yuema137 yuema137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR allows the online processing manager to read the "bootstrax_config" from db_daq so that the targets and post_process in the online processing could be changed dynamically by updating the config in db_daq.

Although we currently don't have auto tests for bootstrax, I believe @cfuselli has tested locally and already makes sure the hard-coded patches here work well for our previous data. Considering that this PR is straight forward and doesn't have any influence on other part of straxen, I think we should merge it soon for the upcoming calibrations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants