Messages on init. #1717

phlptp · 2020-11-11T19:13:37Z

Summary

If merged this pull request will fix a reported bug in received message during the init phase. The iteration worked but no messages were available. This PR adds a test case and fixes the bug.

This PR ended up being a major refactor to resolve the filter timing issues for certain cases. There are no major API changes but the underlying timing code has changed significantly.

…teration

Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com>

…ociated operations

…in the forwarding time coordinator

* add boost to pipelines * Automated formatting of repo files Co-authored-by: Philip Top <phlptp@gmail.com> Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com>

nightlark · 2021-03-16T07:50:09Z

src/helics/core/BrokerBase.cpp

+    // if (m.dest_id == global_federate_id(131074) && m.source_id == global_federate_id(1879048194))
+    // {
+    ///   if (m.action() == CMD_TIME_REQUEST && m.actionTime>timeZero) {
+    //       printf("adding action message\n");
+    //  }
+    //}


Looks like test code to remove eventually.

I am going to leave this in for a little while yet, it won't be in the release code

nightlark · 2021-03-16T08:01:49Z

tests/helics/application_api/FilterAdditionalTests.cpp

+    // std::this_thread::sleep_for(std::chrono::milliseconds(2000));
+    // auto res = broker->query("root", "global_time_debugging");
+    // std::this_thread::sleep_for(std::chrono::milliseconds(2000));
+    // std::this_thread::yield();
+    // auto res2 = broker->query("root", "global_time_debugging");


Maybe something else to remove when not needed, along with a few other similar comments in other test files.

tests/helics/test_files/example_filters.json

nightlark · 2021-03-16T08:14:11Z

src/helics/core/FilterFederate.cpp

+
+    auto cid = filt->core_id;
+    auto* retTarget = filt.get();
+    // auto actualKey = key;


Comments to clean up later?

nightlark · 2021-03-16T08:17:12Z

src/helics/core/FilterFederate.cpp

+            assert(false);
+            // this is an odd condition (not sure what to do yet)


Is this something that should be a TODO?

Is there a way to recover or gracefully shutdown so assert isn't needed?

A lot of the FilterFederate code is pulled from the CommonCore so this just got copied over. I don't think we have reached that condition so I haven't dealt with it yet.

Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com>

* Tests and some fixes for getting messages in initialization mode on iteration * Automated formatting of repo files (#1718) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * add additional test * fix timing bug on local federates * Automated formatting of repo files (#1720) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * fix clang-tidy issue * Automated formatting of repo files (#1722) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * add some capabilities to the dependency management * more tweaks to the time control for ForwardingTimeCoordinator and associated operations * add update back in * some more tweaks for timing tests * make sure the halted time is consistent * make sure the halted time is consistent * update more tests * make the callback locked for endpoint callbacks * fix the echo test and possible inconsistencies in message delivery * try cleaning up a few more tests * add some more tests to echo tests * try a slightly different minimum modification * Try some additional conditions on the Forwarding time coordinators * try another tactic for generating the min federate * try another tactic for generating the min federate * add some delay capability to test comms and some other test function in the forwarding time coordinator * start moving the filters to a separate federate * updates to code for archival * Work out details in FilterFederate and time coordinators * Fix 5 message tests * pass the iteration tests * fix restricted time policy code again * fix compile errors * Try changing the destructor for FilterFederate * fix uninitialized value * more sanitizer fixes * Fix cppcheck issues * add some debugging output for deadlocks on iteration tests * try adjusting when messages get sent * add some time to checks and add boost 1.75 inclusion * Automated formatting of repo files (#1737) * add boost to pipelines * Automated formatting of repo files Co-authored-by: Philip Top <phlptp@gmail.com> Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * add some delays in recorder * Apply suggestions from code review Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com>

* Messages on init. (#1717) * Tests and some fixes for getting messages in initialization mode on iteration * Automated formatting of repo files (#1718) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * add additional test * fix timing bug on local federates * Automated formatting of repo files (#1720) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * fix clang-tidy issue * Automated formatting of repo files (#1722) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * add some capabilities to the dependency management * more tweaks to the time control for ForwardingTimeCoordinator and associated operations * add update back in * some more tweaks for timing tests * make sure the halted time is consistent * make sure the halted time is consistent * update more tests * make the callback locked for endpoint callbacks * fix the echo test and possible inconsistencies in message delivery * try cleaning up a few more tests * add some more tests to echo tests * try a slightly different minimum modification * Try some additional conditions on the Forwarding time coordinators * try another tactic for generating the min federate * try another tactic for generating the min federate * add some delay capability to test comms and some other test function in the forwarding time coordinator * start moving the filters to a separate federate * updates to code for archival * Work out details in FilterFederate and time coordinators * Fix 5 message tests * pass the iteration tests * fix restricted time policy code again * fix compile errors * Try changing the destructor for FilterFederate * fix uninitialized value * more sanitizer fixes * Fix cppcheck issues * add some debugging output for deadlocks on iteration tests * try adjusting when messages get sent * add some time to checks and add boost 1.75 inclusion * Automated formatting of repo files (#1737) * add boost to pipelines * Automated formatting of repo files Co-authored-by: Philip Top <phlptp@gmail.com> Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * add some delays in recorder * Apply suggestions from code review Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com> * Fix offset timing bug (#1759) * Fix offset timing bug * Automated formatting of repo files (#1760) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * Barrier check (#1788) * try to fix a bug in the time barrier clear operation * rework the minFed check for timing * Automated formatting of repo files (#1789) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix clang tidy issues * message required destinations (#1755) * fix some warnings and make it possible to set default for messages to be required to have a destination * Automated formatting of repo files (#1756) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> * fix clan tidy issues * add the option of ordered queries (#1793) * add the option of ordered queries which are slower but have more guarantees of order since they travel on the normal priority paths * Automated formatting of repo files [skip ci] (#1794) * Install Boost for Windows GitHub Actions Workflows (#1790) * Install Boost 1.74 for Windows release build packaging and benchmark installer GitHub Actions workflows * Comment and minor clean-up of the scripts used for the Windows GHA workflows * Automated formatting of repo files Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com> Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com> * update the new command names * Automated formatting of repo files (#1795) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com> * add ordering argument to brokerApp query calls * Automated formatting of repo files (#1796) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix bug causing an overwritten flag * Automated formatting of repo files (#1800) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com> * Event triggered flag (#1804) * Add an event triggered flag for timing, mainly for filters but can be used elsewhere. * make some fixes for recorder and echo * fix clang-tidy warnings * shift the clone tests and more clang-tidy fixes * add flag string translation support * Automated formatting of repo files (#1805) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add delay to clone queries * add delay to clone queries * Automated formatting of repo files (#1806) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix potential deadlock related to query changes * Automated formatting of repo files (#1807) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix clone tests * Automated formatting of repo files (#1808) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add query_reply_ordered * Automated formatting of repo files (#1809) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix time coordinator undefined behavior possibilities * Automated formatting of repo files (#1810) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix a test that was failing under the no-zmq condition (#1812) * Fix a test that was failing under the no-zmq condition * Automated formatting of repo files (#1813) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * now compiling * all tests passing * Automated formatting of repo files (#1821) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * update queryMode style * update the helics.h backup files * Automated formatting of repo files (#1822) Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com> Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com>

phlptp added the bug Issue concerning incorrect code operation label Nov 11, 2020

phlptp requested a review from eberleim November 11, 2020 19:13

eberleim approved these changes Nov 12, 2020

View reviewed changes

phlptp force-pushed the message_on_init branch from e27facc to 2eceea0 Compare November 13, 2020 01:28

phlptp force-pushed the message_on_init branch from f72cf45 to e4258cb Compare December 29, 2020 21:45

phlptp force-pushed the message_on_init branch from aa8b432 to 208b360 Compare March 10, 2021 03:58

phlptp and others added 24 commits March 10, 2021 10:57

Tests and some fixes for getting messages in initialization mode on i…

095da2e

…teration

Automated formatting of repo files (#1718)

15dfc80

Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com>

add additional test

0470bb6

fix timing bug on local federates

281fbdc

Automated formatting of repo files (#1720)

e361aef

Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com>

fix clang-tidy issue

83261a7

Automated formatting of repo files (#1722)

31fa00b

Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com>

add some capabilities to the dependency management

c54d4d9

more tweaks to the time control for ForwardingTimeCoordinator and ass…

3e79912

…ociated operations

add update back in

f3c05fb

some more tweaks for timing tests

f849a93

make sure the halted time is consistent

865229a

make sure the halted time is consistent

9c66da4

update more tests

e858814

make the callback locked for endpoint callbacks

7e31fb7

fix the echo test and possible inconsistencies in message delivery

924de88

try cleaning up a few more tests

e6d0ce4

add some more tests to echo tests

f0afcfe

try a slightly different minimum modification

20677bd

Try some additional conditions on the Forwarding time coordinators

c0d2750

try another tactic for generating the min federate

5229fe7

try another tactic for generating the min federate

1c01905

add some delay capability to test comms and some other test function …

4ce44b2

…in the forwarding time coordinator

start moving the filters to a separate federate

3c376dc

phlptp added 3 commits March 10, 2021 10:57

fix compile errors

caca140

Try changing the destructor for FilterFederate

0139d26

fix uninitialized value

5aa669c

phlptp force-pushed the message_on_init branch 3 times, most recently from df59b99 to 0b54231 Compare March 10, 2021 21:23

more sanitizer fixes

5092c54

phlptp force-pushed the message_on_init branch from 0b54231 to 5092c54 Compare March 10, 2021 22:56

Fix cppcheck issues

f1f2c46

phlptp requested a review from nightlark March 10, 2021 23:54

add some debugging output for deadlocks on iteration tests

aa2795c

phlptp force-pushed the message_on_init branch 5 times, most recently from 3a4d0b2 to 8213588 Compare March 12, 2021 01:17

try adjusting when messages get sent

6889dd7

phlptp force-pushed the message_on_init branch from 8213588 to 6889dd7 Compare March 15, 2021 15:51

add some time to checks and add boost 1.75 inclusion

a2c0bf2

phlptp force-pushed the message_on_init branch 2 times, most recently from 7b03c60 to a2c0bf2 Compare March 15, 2021 18:13

github-actions bot and others added 2 commits March 15, 2021 13:21

Automated formatting of repo files (#1737)

5c16320

* add boost to pipelines * Automated formatting of repo files Co-authored-by: Philip Top <phlptp@gmail.com> Co-authored-by: Philip Top <top1@llnl.gov> Co-authored-by: HELICS-bot <HELICS-bot@users.noreply.github.com>

add some delays in recorder

51424f9

phlptp force-pushed the message_on_init branch from 4cfa242 to 51424f9 Compare March 15, 2021 21:39

nightlark approved these changes Mar 16, 2021

View reviewed changes

Apply suggestions from code review

2f97df0

Co-authored-by: Ryan Mast <nightlark@users.noreply.github.com>

phlptp merged commit 409f5b3 into develop Mar 16, 2021

phlptp deleted the message_on_init branch March 16, 2021 18:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Messages on init. #1717

Messages on init. #1717

phlptp commented Nov 11, 2020 •

edited

Loading

nightlark Mar 16, 2021

phlptp Mar 16, 2021

nightlark Mar 16, 2021

nightlark Mar 16, 2021

nightlark Mar 16, 2021

phlptp Mar 16, 2021

		assert(false);
		// this is an odd condition (not sure what to do yet)

Messages on init. #1717

Messages on init. #1717

Conversation

phlptp commented Nov 11, 2020 • edited Loading

Summary

nightlark Mar 16, 2021

Choose a reason for hiding this comment

phlptp Mar 16, 2021

Choose a reason for hiding this comment

nightlark Mar 16, 2021

Choose a reason for hiding this comment

nightlark Mar 16, 2021

Choose a reason for hiding this comment

nightlark Mar 16, 2021

Choose a reason for hiding this comment

phlptp Mar 16, 2021

Choose a reason for hiding this comment

phlptp commented Nov 11, 2020 •

edited

Loading