Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add features to history output #1024

Merged
merged 24 commits into from
Apr 9, 2024
Merged

Add features to history output #1024

merged 24 commits into from
Apr 9, 2024

Conversation

brryan
Copy link
Collaborator

@brryan brryan commented Mar 14, 2024

PR Summary

We want additional control in how history outputs are produced. This PR:

  • Adds the ability to enroll history outputs on a per-package basis
  • Adds the ability to have multiple history output files with different output quantities and different cadences [breaking change by changing e.g. my_output.hst to my_output.out2.hst]
  • Merges MPI reductions by operation to reduce communication
  • Adds the ability to enroll vector quantities as history outputs.
  • Include example vector output in advection example.

PR Checklist

  • Code passes cpplint
  • New features are documented.
  • Adds a test for any bugs fixed. Adds tests for new features.
  • Code is formatted
  • Changes are summarized in CHANGELOG.md
  • Change is breaking (API, behavior, ...)
    • Change is additionally added to CHANGELOG.md in the breaking section
    • PR is marked as breaking
    • Short summary API changes at the top of the PR (plus optionally with an automated update/fix script)
  • CI has been triggered on Darwin for performance regression tests.
  • Docs build
  • (@lanl.gov employees) Update copyright on changed files

@brryan brryan changed the title WIP: Add features to history output Add features to history output Mar 14, 2024
example/advection/parthinput.advection Show resolved Hide resolved
src/outputs/history.cpp Show resolved Hide resolved
@@ -147,9 +149,20 @@ struct HistoryOutputVar {
: hst_op(hst_op_), hst_fun(hst_fun_), label(label_) {}
};

struct HistoryOutputVec {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I treated vector output separately from scalar history output because I didn't see another way to do this apart from a breaking change where enrolled history functions only return std::vector<Real> i.e. all existing history implementations would need to switch.

src/outputs/history.cpp Outdated Show resolved Hide resolved
Copy link
Collaborator

@Yurlungur Yurlungur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this. This is a really nice quality of life improvement. A few nitpicks below, but nothing blocking.

example/advection/advection_package.cpp Outdated Show resolved Hide resolved
example/advection/advection_package.cpp Show resolved Hide resolved
src/outputs/history.cpp Show resolved Hide resolved
src/outputs/history.cpp Outdated Show resolved Hide resolved
src/outputs/history.cpp Show resolved Hide resolved
src/outputs/history.cpp Outdated Show resolved Hide resolved
Copy link
Collaborator

@pdmullen pdmullen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

example/advection/parthinput.advection Show resolved Hide resolved
src/outputs/history.cpp Show resolved Hide resolved
src/outputs/outputs.cpp Show resolved Hide resolved
Copy link
Collaborator

@pgrete pgrete left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice change. Thanks!

example/advection/advection_package.cpp Show resolved Hide resolved
example/advection/advection_package.cpp Show resolved Hide resolved
Comment on lines +69 to +72
for (const auto &pkg_name : requested_packages) {
const auto &it = all_packages.find(pkg_name);
if (it != all_packages.end()) {
packages[(*it).first] = (*it).second;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we throw an error if the packages has not been found? To prevent accidental typos in the input file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I agree. One could imagine someone wanting to write out history from specific packages and automatically not getting that output if they enable/disable individual packages at runtime, but I think catching typos is more important. Also, a way to disable packages at runtime is to enroll an empty package with the same name, which would work for the above.

Changed to error if this if statement is false.

Comment on lines 128 to 130
["power0", 7.06177e-02, 1.39160e-02],
["power1", 3.88112e-02, 2.59597e-03],
["power2", 2.65948e-02, 7.19427e-04],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be advected_powers#, isn't it?
Also, if there are no objection, I'd prefer the indices separated by _ as we're using that kind of convention in other places already.

Copy link
Collaborator Author

@brryan brryan Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the other labels were actually wrong as well (e.g. total should have been total_advected). I changed all the variable names to be corrected and also added a test of the column headers.

And I also added the _ by default for vector-length history output so it is now e.g. advected_powers_2.

IN ADDITION I also enforced a space between each column so we can parse the labels more easily. In develop currently we can get headers like # [0]=time [1]=dt [2]=my_long_var_name[3]=my_other_longvar_name which can be annoying to parse via something simple like .split() in python. I don't think this is a breaking change because whitespace between variable names was already a possibility (for short variable names) but I should mention it.

@pgrete pgrete mentioned this pull request Mar 26, 2024
@brryan
Copy link
Collaborator Author

brryan commented Apr 2, 2024

@pgrete @Yurlungur @pdmullen My plan is to set this to automerge sometime tomorrow -- last call for objections!

Thanks for your reviews all

@brryan brryan enabled auto-merge (squash) April 3, 2024 14:13
Comment on lines +90 to 92
<parthenon/output5>
file_type = ascent
dt = -0.05 # soft disabled by default, as Ascent is an optional dependency
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is what's causing the failed regression test as ascent outputs are manually enabled in .github/workflows/ci-extended.yml via parthenon/output4/dt=0.05

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 thank you for noticing this @pgrete!

I also modified .github/workflows/ci-extended.yml to give an error message if test fails.

@lroberts36 lroberts36 disabled auto-merge April 8, 2024 22:27
@brryan brryan enabled auto-merge (squash) April 9, 2024 02:26
@brryan brryan disabled auto-merge April 9, 2024 19:33
@brryan brryan enabled auto-merge (squash) April 9, 2024 19:34
@brryan brryan merged commit d3f5b6a into develop Apr 9, 2024
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants