Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating a report after the execution of workflow. #975

Closed
BoPeng opened this issue May 19, 2018 · 2 comments
Closed

Generating a report after the execution of workflow. #975

BoPeng opened this issue May 19, 2018 · 2 comments

Comments

@BoPeng
Copy link
Contributor

BoPeng commented May 19, 2018

Many workflow systems (e.g. nextflow and snakemake) generates a report after the completion of a workflow, including all the information such as steps executed, CPU/MEM usages, time, perhaps information on intermediate files. It is nice to have because it keeps all information in one place and users can refer back to the report if needed. SoS already has most of the information at hand and it would be nice to offer an option to create a report.

Hard to get name for an option though because we are limited to single letter options for sos run. -r is already used by remote execution.

@BoPeng
Copy link
Contributor Author

BoPeng commented Jun 4, 2018

@gaow I am still working on this ticket but the framework is done and ready for comment. More specifically,

  1. report.py is added to collect workflow runtime information during execution. Right now it only collects these.
  2. A jinja2 templste is added to show the information.

Right now the template is very crude and I am adding CSS, but more importantly we need to decide what to display in the report. I am thinking of

  1. start/end/duration of tasks.
  2. tags for tasks because they relate tasks to step/ID etc
  3. CPU/MEM usage of tasks because they are available.

I am not sure if it is needed to list the executed steps, input and output files etc.

This feature re-uses the reporting mechanism used for pack/unpack but is no longer compatible with them. pack/unpack are therefore removed, at least for now.

BoPeng pushed a commit that referenced this issue Jun 4, 2018
@BoPeng
Copy link
Contributor Author

BoPeng commented Jun 4, 2018

Roughly done.

@BoPeng BoPeng closed this as completed Jun 4, 2018
BoPeng pushed a commit that referenced this issue Jun 4, 2018
BoPeng pushed a commit to vatlab/sos-notebook that referenced this issue Jun 4, 2018
BoPeng pushed a commit to vatlab/sos-docs that referenced this issue Jun 4, 2018
BoPeng pushed a commit that referenced this issue Jun 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant