Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: create a script to obtain the status of the simulation #117

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Namyalg
Copy link
Member

@Namyalg Namyalg commented May 25, 2022

Pull Request Template

Description

The existing endpoints for the gcbm example do not report a failure in the simulation.
The script simulation-status.py returns the logs of the docker container (if it is running).
This can be used in the process of automated testing

To obtain the logs of the simulation :

  1. Start the simulation
  2. Run the script simulation-status.py , a text file containing the logs will be created

Fixes #93

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

aornugent
aornugent previously approved these changes May 29, 2022
@aornugent
Copy link
Contributor

I think in order to make this an integration test, we need to publish a reference to compare against. Can you add the scaffolding to save the results and compare with another file of the same schema? Then we check the reference into version control and monitor change from there.

This does not tell us whether the results are numerically identical, which is considered the holy grail in integration tests.

@Namyalg
Copy link
Member Author

Namyalg commented Jun 7, 2022

A quick demonstration on how the script can be used

logs.mp4

@Namyalg
Copy link
Member Author

Namyalg commented Jun 8, 2022

Logs from a successful simulation :

docker_logs.txt

@aornugent aornugent requested a review from radistoubalidis June 8, 2022 04:08
@aornugent
Copy link
Contributor

aornugent commented Jun 8, 2022

@radistoubalidis - could you use this script in your CML Action?

@Namyalg - if it weren't for the timestamps, we would be able to use the hash of these logs as a reference. Are timestamps included in the console output when you run the simulation inside the container interactively? Or are they part of th Docker output?

@radistoubalidis
Copy link
Contributor

@radistoubalidis - could you use this script in your CML Action?

@Namyalg - if it weren't for the timestamps, we would be able to use the hash of these logs as a reference. Are timestamps included in the console output when you run the simulation inside the container interactively? Or are they part of th Docker output?

I just added it to my action here , the only issue is a haven't yet managed to run the whole workflow inside actions so for now, I made it use the results from a sim I run locally.

@Namyalg
Copy link
Member Author

Namyalg commented Jun 8, 2022

The timestamp can be removed, I'll process it such that only

@radistoubalidis - could you use this script in your CML Action?

@Namyalg - if it weren't for the timestamps, we would be able to use the hash of these logs as a reference. Are timestamps included in the console output when you run the simulation inside the container interactively? Or are they part of th Docker output?

Yes, this can be removed, I'll process it in such a way that <2022-06-08 00:16:24.906554> (info) - Config has files: 7 will be stored as Config has files: 7

To identify the simulation uniquely, how about appending the end of the file with a randomly generated string?

Copy link
Contributor

@radistoubalidis radistoubalidis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I managed to add the python script to my simulation results and retrieve the log file

@Namyalg Namyalg marked this pull request as ready for review July 3, 2022 04:20
@aornugent
Copy link
Contributor

@Namyalg - can you please remind me if this is ready to merge? Do you have permissions to do so?

@Namyalg
Copy link
Member Author

Namyalg commented Dec 24, 2022

Yes @aornugent, I tested the script again, it works as expected -> retrieve the logs of a container running the simulation
It is ready for merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GCBM Test suite does not report simulation failure
3 participants