Optimized dispatches from HERON to HYBRID #202

BotrosHanna-INL · 2022-08-15T23:25:21Z

Pull Request Description

What issue does this change request address?

Issue #201 : Using HERON OptimizedDispatch in HYBRID

What are the significant changes in functionality due to this change request?

HERON-HYBRID Integration

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

1. Review all computer code.
2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
4. Automated Tests should pass.
5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large tes.
6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
8. If an analytic test is changed/added, the the analytic documentation must be updated/added.
9. If any test used as a basis for documentation examples have been changed, the associated documentation must be reviewed and assured the text matches the example.

BotrosHanna-INL · 2022-08-15T23:32:12Z

@PaulTalbot-INL
Hi Paul,
Here is the (dispatches from HERON to HYBRID) PR for your comments.

PaulTalbot-INL · 2022-08-17T14:33:00Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/create_user_input.py

+  Creates a list of components' capacities from the HERON inpue file plus creating placeholders for the components' capacities in HYRBID
+  @ In, heron input, str, the HERON input XML file
+  @ In, output_file, str, the text file where the list of components capacities should be printed.
+  @ Out, output_file, str, text a file autoloaded with the list of components' capacities from the HERON file and placeholders for the components' capacities in HYRBID


these docstrings should be indented one more time beyond the triple quotes

coming back and checking, this method also doesn't return anything, so out should be None

PaulTalbot-INL · 2022-08-17T14:33:39Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/create_user_input.py

+  """
+
+  # Creating the HERON XML file tree
+  HERON_inp_tree = ET.parse(heron_input)


This is fine for now, but we may not always use XML and we should use the RAVEN tree reader instead. We can fix that if and when we ever get that far.

PaulTalbot-INL · 2022-08-17T14:34:36Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/create_user_input.py

+  Creates a list of dispatches (components' optimized variables) from the HERON inpue file plus creating placeholders for the dispatches in HYRBID
+  @ In, dispatch_print, str, the distpach csv file with a list of optimized variables
+  @ In, output_file, str, the text file where the list of optimized dispatches should be printed.
+  @ Out, output_file, str, a text file autoloaded with the optimized dispatches plus placeholders for the HYBRID dispatches


same indent needed here, 2 more spaces past the triple quotes

also, this method doesn't look to me like it returns anything, so this should be @ Out, None

PaulTalbot-INL · 2022-08-17T14:36:21Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/create_user_input.py

+from xml.etree import ElementTree as ET
+import pandas as pd
+from colorama import Fore
+from colorama import Style


is colorama part of our standard install?

changed it.

PaulTalbot-INL · 2022-08-17T14:37:42Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/export2Hybrid.py

+  Identifies which component dispatch/capacity in HYBRID corresponds to which component dispatch/capacity in HERON.
+  @ In, user-input file, str, the user-input text file
+  @ Out, a dictionary that matches HERON and HYBRID capacities, dict, Defining the HYBRID capacity that corresponds to each HERON capacity
+  @ Out, a dictionary that matches HERON and HYBRID dispatches, dict, Defining the HYBRID dispatch that corresponds to each HERON dispatch


the first argument here should be the variable name, so heron_hybrid_dispatch for the second output.

I know that giving the name of an output makes no sense, but them's the rules.

PaulTalbot-INL · 2022-08-17T14:38:24Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/export2Hybrid.py

+  """
+  Extracts the most interesting dataset from the a CSV file that we get from HERON. This CSV file includes optimized dispatchs that are calaculated at different years and samples.
+  @ In, dispatch_print, str, a csv file produced by HERON and includes optimized components' time-dependent variables at different years, different samples
+  @ Out, interesting dataset, pandas.core.frame.DataFrame, time-dependent optimized variables


the methods in this file also need docstrings indented within the triple quotes 1 more level

PaulTalbot-INL · 2022-08-17T14:39:33Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/tests

+  [./OptimizedDispatch_FromHERON2HYBRID]
+    type = RavenPython
+    input = 'export2Hybrid.py heron_input.xml dispatch_print.csv'
+  [../]


is this missing the test to create the dispatch_print.csv?

I added a test to create dispatch_print.csv

BotrosHanna-INL · 2022-08-22T20:28:28Z

@PaulTalbot-INL

Thank you, Paul, for you comments. Let me know if I need to make more changes. I know that I still need to add this work to the user manual. I will send another message after I do it.

PaulTalbot-INL · 2022-08-22T20:34:05Z

tests/integration_tests/mechanics/optimizedDispatch2Hybrid/tests

+      [./createUserInput]
+        type = RavenPython
+        input = 'create_user_input.py heron_input.xml Debug_Run_o/dispatch_print.csv'
+        [./dispatches2Hybrid]


Does nesting these work? I've never tried that.

It looks like it only ran the one test:

... (16/32) Success( 11.24sec)/home/civet/civet/build_0/HERON/tests/integration_tests/mechanics/var_demand_fix_price/VarDemandFixPrice (17/32) Success( 11.86sec)/home/civet/civet/build_0/HERON/tests/integration_tests/mechanics/var_demand_var_price/VarDemandVarPrice (18/32) Success( 11.39sec)/home/civet/civet/build_0/HERON/tests/integration_tests/mechanics/optimizedDispatch2Hybrid/DebugMode (19/32) Success( 3.74sec)/home/civet/civet/build_0/HERON/tests/integration_tests/workflows/MOPED/consumption/Moped_consumption (20/32) Success( 20.24sec)/home/civet/civet/build_0/HERON/tests/integration_tests/workflows/MOPED/simple/Moped_simple (21/32) Success( 3.57sec)/home/civet/civet/build_0/HERON/tests/integration_tests/workflows/MOPED/storage/Moped_storage ...

I think you want to separate the tests, and set up prerequisites.

Thank you! I thought that nesting the tests sets up the prerequisites and since I did not get any error messages, I thought that the RavenPython tests are successful. Now, I set up the prerequisites and I think all the tests are run now.

PaulTalbot-INL

LGTM

PaulTalbot-INL · 2022-08-23T14:12:59Z

I don't see the user manual changes, but I do see the readme. Do you still have user manual changes coming?

BotrosHanna-INL and others added 8 commits April 13, 2022 13:43

tests/integration_tests/mechanics/hybrid_heron_integration/

a77478a

Merge branch 'idaholab:devel' into devel

a108833

Merge branch 'idaholab:devel' into devel

f0f0638

Merge branch 'idaholab:devel' into devel

8b3b17b

optimized dispatches from HERON to HYBRID

7f07e81

deleting extra folder

6e09018

tests file

01a37b9

remove extra file

b61be3f

BotrosHanna-INL changed the title ~~Develop~~ Optimized dispatches from HERON to HYBRID Aug 15, 2022

BotrosHanna-INL requested a review from PaulTalbot-INL August 15, 2022 23:28

PaulTalbot-INL requested changes Aug 17, 2022

View reviewed changes

reviewers' comments

ea0c4b3

PaulTalbot-INL reviewed Aug 22, 2022

View reviewed changes

fixing the tests

f977ba4

PaulTalbot-INL previously approved these changes Aug 23, 2022

View reviewed changes

user manual section

ecd798d

BotrosHanna-INL dismissed PaulTalbot-INL’s stale review via ecd798d August 23, 2022 19:27

PaulTalbot-INL previously approved these changes Aug 23, 2022

View reviewed changes

fixing typos

e56e316

BotrosHanna-INL dismissed PaulTalbot-INL’s stale review via e56e316 August 23, 2022 19:40

PaulTalbot-INL previously approved these changes Aug 23, 2022

View reviewed changes

BotrosHanna-INL added 2 commits August 23, 2022 14:03

fixing more typos

d60bde1

fixing more typos

32a6748

BotrosHanna-INL dismissed PaulTalbot-INL’s stale review via 32a6748 August 23, 2022 20:03

PaulTalbot-INL approved these changes Aug 23, 2022

View reviewed changes

PaulTalbot-INL merged commit e03b1ab into idaholab:devel Aug 23, 2022

PaulTalbot-INL mentioned this pull request Sep 1, 2022

Using HERON OptimizedDispatch in HYBRID #201

Closed

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimized dispatches from HERON to HYBRID #202

Optimized dispatches from HERON to HYBRID #202

BotrosHanna-INL commented Aug 15, 2022 •

edited by PaulTalbot-INL

Loading

BotrosHanna-INL commented Aug 15, 2022

PaulTalbot-INL Aug 17, 2022

PaulTalbot-INL Aug 17, 2022

BotrosHanna-INL Aug 22, 2022

PaulTalbot-INL Aug 17, 2022

BotrosHanna-INL Aug 22, 2022

PaulTalbot-INL Aug 17, 2022

PaulTalbot-INL Aug 17, 2022

BotrosHanna-INL Aug 22, 2022

PaulTalbot-INL Aug 17, 2022

BotrosHanna-INL Aug 22, 2022

PaulTalbot-INL Aug 17, 2022

BotrosHanna-INL Aug 22, 2022

PaulTalbot-INL Aug 17, 2022

BotrosHanna-INL Aug 22, 2022

PaulTalbot-INL Aug 17, 2022

BotrosHanna-INL Aug 22, 2022

BotrosHanna-INL commented Aug 22, 2022

PaulTalbot-INL Aug 22, 2022

PaulTalbot-INL Aug 22, 2022

BotrosHanna-INL Aug 22, 2022

PaulTalbot-INL left a comment

PaulTalbot-INL commented Aug 23, 2022

Optimized dispatches from HERON to HYBRID #202

Optimized dispatches from HERON to HYBRID #202

Conversation

BotrosHanna-INL commented Aug 15, 2022 • edited by PaulTalbot-INL Loading

Pull Request Description

What issue does this change request address?

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

BotrosHanna-INL commented Aug 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BotrosHanna-INL commented Aug 22, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PaulTalbot-INL left a comment

Choose a reason for hiding this comment

PaulTalbot-INL commented Aug 23, 2022

BotrosHanna-INL commented Aug 15, 2022 •

edited by PaulTalbot-INL

Loading