Skip to content

Commit

Permalink
feat: improve logging when parsing commands from workflow description
Browse files Browse the repository at this point in the history
  • Loading branch information
DongzeHE committed Apr 7, 2023
1 parent 2d09dcd commit 0d91772
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 5 deletions.
6 changes: 3 additions & 3 deletions docs/source/get-workflow-config-command.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ Providing information in workflow configuration file

Usually, a published workflow contains four sections:

1) ``Recommended Simpleaf Configuration``: For most users, this section is the only section needed to be completed, i.e., replacing all ``null`` in the section by a meaningful value. The Jsonnet program should be smart enough to generate a valid workflow description JSON from the information provided here.
2) ``Optional Simpleaf Configuration``: This section contains advanced options that are not covered in the ``Recommended Simpleaf Configuration`` section. The behavior of the workflow can be finely tuned by providing information in this section.
1) ``Recommended Configuration``: For most users, this section is the only section needed to be completed, i.e., replacing all ``null`` in the section by a meaningful value. The Jsonnet program should be smart enough to generate a valid workflow description JSON from the information provided here.
2) ``Optional Configuration``: This section contains advanced options that are not covered in the ``Recommended Configuration`` section. The behavior of the workflow can be finely tuned by providing information in this section.
3) ``External Commands``: This section contains all external shell command records. If you see ``TBD`` in this section, it means that these fields will be filled by the Jsonnet program automatically.
4) ``meta_info``: This section contains the meta info of this workflow. Sometimes the information provided here is used for controling global arguments, for example the output directory and the number of threads used for each invoked command.

For most users, the ``Recommended Simpleaf Configuration`` is the only section needed to be completed to allow the Jsonnet program to generate a valid workflow description JSON. To fill the missing information, one just needs to replace the ``null`` with a meaningful value. **Notice that** to ease the later parsing process, the values of all command arguments must be provided as strings, i.e., wrapped by quotes (``"value"``), even for integers like the number of threads (for example, ``{“--threads”: "16"}`` for simpleaf commands).
For most users, the ``Recommended Configuration`` is the only section needed to be completed to allow the Jsonnet program to generate a valid workflow description JSON. To fill the missing information, one just needs to replace the ``null`` with a meaningful value. **Notice that** to ease the later parsing process, the values of all command arguments must be provided as strings, i.e., wrapped by quotes (``"value"``), even for integers like the number of threads (for example, ``{“--threads”: "16"}`` for simpleaf commands).

For example, the complete ``meta_info`` section in the configuration Jsonnet program of the ``cite-seq-ADT+HTO_10xv2`` workflow should looks like the following (with all comments removed).

Expand Down
17 changes: 15 additions & 2 deletions src/utils/workflow_utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,13 @@ impl SimpleafWorkflow {
// file are strings.
if pn.is_external() {
// creating an external command records using the args recorded in the field
let external_cmd = pn.create_external_cmd(field)?;
let external_cmd =
match pn.create_external_cmd(field) {
Ok(v) => v,
Err(e) => {
bail!("Could not parse external command {} for Step {}. The error message was: {}", pn, step, e);
}
};

cmd_queue.push(CommandRecord {
step,
Expand All @@ -258,7 +264,14 @@ impl SimpleafWorkflow {
});
} else {
// create a simpleaf command record using the args recorded in the field
let simpleaf_cmd = pn.create_simpleaf_cmd(field)?;
let simpleaf_cmd =
match pn.create_simpleaf_cmd(field) {
Ok(v) => v,
Err(e) => {
bail!("Could not parse simpleaf command {} for Step {}. The error message was: {}", pn, step, e);
}
};


cmd_queue.push(CommandRecord {
step,
Expand Down

0 comments on commit 0d91772

Please sign in to comment.