Incremental workflow evaluation #946
Conversation
Codecov Report
@@ Coverage Diff @@
## master #946 +/- ##
==========================================
+ Coverage 86.35% 87.58% +1.23%
==========================================
Files 80 81 +1
Lines 9078 9183 +105
==========================================
+ Hits 7839 8043 +204
+ Misses 1239 1140 -99 |
| result_op.connect(4, ds) | ||
| new_end_op = splitter.split(chunk_size=2, scoping=scoping) | ||
|
|
||
| min2 = new_end_op.get_output(0, dpf.types.field) |
There was a problem hiding this comment.
Does new_end_op have any generated outputs (to do new_end_op.outputs....()) ?
src/ansys/dpf/core/incremental.py
Outdated
| # last operator should support incremental evaluation | ||
| # but it should be permissive too (bad doc/spec whatever) |
There was a problem hiding this comment.
(for bad docs or specs?) Not sure what the content in this bracket means.
There was a problem hiding this comment.
Yes, it was unclear.
Basically this function is used for validation, but in the case this function doesn't validate, it should not create an error but simply inform the user through a warning.
Because the operator that it is trying to validate may have a bad specification or documentation or whatever the inconsistency is.
Please tell me if the current comments are clearer in that regard.
Co-authored-by: JennaPaikowsky <98607744+JennaPaikowsky@users.noreply.github.com>
|
Hi @ansys-akarcher, |
|
Yes, I need to:
|
…t/incremental_eval
|
|
||
|
|
||
| # Obtain results on the same pin numbers | ||
| min = new_end_op.get_output(0, dpf.types.field) |
There was a problem hiding this comment.
@ansys-akarcher same remark as @cbellot000:
can't we write
min = new_end_op.outputs.min_field.eval()
or something similar?
Co-authored-by: Camille Bellot <80476446+cbellot000@users.noreply.github.com>
These changes brings a simple API around the for_each and chunk_in_for_each_range operators.
The goal is to be able to process very huge files with minimal effort, including being able to let the API choose for ourselves the chunk_size.
Also, this draft PR needs the changes from this one beforehand