Sim3 GA interface

[Nhatkhang]

---

Pull Request Description

#1956 Adding Simulate3 - GA interface to RAVEN
Update the CodeInterfaces class for reading in/out of Simulate3 and perform GA on loading pattern optimized problem.
This interface is specifically designed for loading pattern problems only. Therefore, there are still some drawbacks to it. Furthermore, in this current version, due to the fact that GA from raven is applied to the single objective optimized problem, a dummy variable is created in the interface/data in an attempt to convert multi-objective to a single objective problem.

khnguy22-NCSU ARDOR team

This request review is specially assigned to @wangcj05

---

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

• 1. Review all computer code.
• 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
• 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
• 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
• 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
• 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
• 7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
• 8. If an analytic test is changed/added is the the analytic documentation updated/added?
• 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

[wangcj05]

@Nhatkhang @Jimmy-INL Sorry for the late response, it seems there are so many files added to your branch. Could you clean up your branch so that we can review it? The following are my suggestions:

• In ./tests/framework/CodeInterfaceTests/Simulate/Simulate_RAVEN-GA (see following picture), there is no raven test input file added for this PR. Could you add the raven input file to this folder?

• Could you reduce the number of samplings in the test case, there are a lot of files added in ./tests/framework/CodeInterfaceTests/Simulate/Simulate_RAVEN-GA/sampleGA-Simulate/, could you reduce the number to less than 5, and remove the rest of folders and files that you committed to the PR? The purpose is to test the structure of the code interface

After the clean up, I think GitHub will show your changes and we can review it. Let me know if you need any help, and we can schedule a meeting to discuss it.

[Nhatkhang]

@Nhatkhang @Jimmy-INL Sorry for the late response, it seems there are so many files added to your branch. Could you clean up your branch so that we can review it? The following are my suggestions:

• In ./tests/framework/CodeInterfaceTests/Simulate/Simulate_RAVEN-GA (see following picture), there is no raven test input file added for this PR. Could you add the raven input file to this folder?
• Could you reduce the number of samplings in the test case, there are a lot of files added in ./tests/framework/CodeInterfaceTests/Simulate/Simulate_RAVEN-GA/sampleGA-Simulate/, could you reduce the number to less than 5, and remove the rest of folders and files that you committed to the PR? The purpose is to test the structure of the code interface

After the clean up, I think GitHub will show your changes and we can review it. Let me know if you need any help, and we can schedule a meeting to discuss it.

Thank you for your response, I will re-add xml files to that directory and re-run with a smaller sample.
I will keep you updated.

[Nhatkhang]

@Nhatkhang @Jimmy-INL Sorry for the late response, it seems there are so many files added to your branch. Could you clean up your branch so that we can review it? The following are my suggestions:

• In ./tests/framework/CodeInterfaceTests/Simulate/Simulate_RAVEN-GA (see following picture), there is no raven test input file added for this PR. Could you add the raven input file to this folder?
• Could you reduce the number of samplings in the test case, there are a lot of files added in ./tests/framework/CodeInterfaceTests/Simulate/Simulate_RAVEN-GA/sampleGA-Simulate/, could you reduce the number to less than 5, and remove the rest of folders and files that you committed to the PR? The purpose is to test the structure of the code interface

After the clean up, I think GitHub will show your changes and we can review it. Let me know if you need any help, and we can schedule a meeting to discuss it.

@wangcj05 @Jimmy-INL
Hi all,
I have added xml and reduced the sample test to 5, thus there should be 25 subdir in the sample /Simulate_RAVEN-GA/ .
In each of them, they will contain input/output for simulate3 code. You can use them to do test case as you desire.
The most important files to review should be the python script in /ravenframework/CodeInterfaceClasses/SIMULATE3 , namely :
SimulateData.py
SimulateInterface.py
SpecificParser.py

Thank you for your time and patience.

[joshua-cogliati-inl]

Hm, why is this deleting plugins/SRAW?

[Jimmy-INL]

@Nhatkhang did you delete plugins/SRAW?
@joshua-cogliati-inl I think that was by mistake.

[Nhatkhang]

@Nhatkhang did you delete plugins/SRAW?
@joshua-cogliati-inl I think that was by mistake.

No, I did not delete it. The reason probably, I did not sync fork the plugins/STRAW part

[wangcj05]

@Nhatkhang @Jimmy-INL Thanks a lot. Here are the additional steps that are needed:

1. Please add a manual for the simulate in raven user manual
2. For the code interface, could you help me to identify which input files will be perturbed, and which output files will be used to collect output data for RAVEN?
3. Could you also add the RAVEN csv outputs from OutStream in the following graph to this PR?
   

[Nhatkhang]

@Nhatkhang @Jimmy-INL Thanks a lot. Here are the additional steps that are needed:

1. Please add a manual for the simulate in raven user manual
2. For the code interface, could you help me to identify which input files will be perturbed, and which output files will be used to collect output data for RAVEN?
3. Could you also add the RAVEN csv outputs from OutStream in the following graph to this PR?
   

Yes I will do it.
For the 2nd question. The input for simulate3 code is generated based on the specific interface through sim3-param.xml and sim3-perturb.xml in the test directory. Output for Raven to extract data is located in each br*_* subdir, they are all name input.out I believe.

[joshua-cogliati-inl]

@Nhatkhang did you delete plugins/SRAW?
@joshua-cogliati-inl I think that was by mistake.

No, I did not delete it. The reason probably, I did not sync fork the plugins/STRAW part

Um, commit 938fcb0 deletes it.

(Would you like me to make a clean branch that doesn't delete plugins/SRAW?)

[Nhatkhang]

@Nhatkhang did you delete plugins/SRAW?
@joshua-cogliati-inl I think that was by mistake.

No, I did not delete it. The reason probably, I did not sync fork the plugins/STRAW part

Um, commit 938fcb0 deletes it.

(Would you like me to make a clean branch that doesn't delete plugins/SRAW?)

@joshua-cogliati-inl
I really appreciate if you do so. I do not really know exactly why the plugins/STRAW was deleted though.

[wangcj05]

@Nhatkhang Based on your answer to question 2, here are some additional requests:

• The raven input file require additional three files (see the following graph), could you add them to the PR?
  

• If only input.out is used by RAVEN, could you remove the other files in br*_* subdirectories, since they are automatically generated by RAVEN.

[joshua-cogliati-inl]

I created a rebased version of the patch in: https://github.com/joshua-cogliati-inl/raven/tree/sim3-GA-interface-work
To use it:

git remote add joshua-cogliati-inl git@github.com:joshua-cogliati-inl/raven.git
git checkout sim3-GA-interface-work

and then use git push --set-upstream followed by the repository you want to push it to (not mine) the first time you push.

[Nhatkhang]

git checkout sim3-GA-interface-work

@wangcj05 I added xml files and removed un-used file in b*_r* as you requested. I am working on the manual for simulate3 interface.

[joshua-cogliati-inl]

@Nhatkhang It is going to continue to fail precheck until you fix the deletion of SRAW.

[Nhatkhang]

git checkout sim3-GA-interface-work

@joshua-cogliati-inl Sorry for confusing.
I used your instruction but it returned errors shown below:

Please excuse me as I am not fully familiar with git on linux command.

[joshua-cogliati-inl]

Sorry, I forgot a step. git fetch --all

After that the checkout should work.

[joshua-cogliati-inl]

@Nhatkhang I added your two new commits to sim3-GA-interface-work (and yes, learning git is a challenge)

[Jimmy-INL]

the path should be relative to the working directory. Please replace Simulate_RAVEN-GA/ by ./

[Jimmy-INL]

same here

[Jimmy-INL]

same here

[Jimmy-INL]

same here

[Jimmy-INL]

here

[Jimmy-INL]

And finally here

[Jimmy-INL]

Please update the docstring. For instance, should this be simulate instead of Scale

[Jimmy-INL]

Remove this line or update it accordingly

[Nhatkhang]

git remote add joshua-cogliati-inl git@github.com:joshua-cogliati-inl/raven.git

@joshua-cogliati-inl I tried again but I do not see your updated remote branch as follows:

@Jimmy-INL I have done the clean up, after resolving with plugins/STRAW deletion, I will push it to remote.

[joshua-cogliati-inl]

@joshua-cogliati-inl I tried again but I do not see your updated remote branch as follows:

Run:
git fetch --all

After you have the branch, you can transfer any new commits you made to it with git cherry-pick commitid
So for example I ran:
git cherry-pick a49f40b822e5b8f78ccbc3814de9f998abd4c482 to add your latest commit to the branch.

[Nhatkhang]

@joshua-cogliati-inl I tried again but I do not see your updated remote branch as follows:

Run: git fetch --all

After you have the branch, you can transfer any new commits you made to it with git cherry-pick commitid So for example I ran: git cherry-pick a49f40b822e5b8f78ccbc3814de9f998abd4c482 to add your latest commit to the branch.

Hi @joshua-cogliati-inl, I run get fetch --all as you suggested but it seems that I do not have the permission to fecth your branch:

[joshua-cogliati-inl]

Hm, can you get to it via web: https://github.com/joshua-cogliati-inl/raven/tree/sim3-GA-interface-work
If so, try:

git remote remove joshua-cogliati-inl
git remote add  joshua-cogliati-inl https://github.com/joshua-cogliati-inl/raven.git
git fetch --all

[Jimmy-INL]

@Nhatkhang and @wangcj05, does this give the user the option if he did not provide a sequence to default to simulate? Do we really want that?

[Nhatkhang]

Hm, can you get to it via web: https://github.com/joshua-cogliati-inl/raven/tree/sim3-GA-interface-work If so, try:

git remote remove joshua-cogliati-inl
git remote add  joshua-cogliati-inl https://github.com/joshua-cogliati-inl/raven.git
git fetch --all

Thank you for your patience, @joshua-cogliati-inl . I successfully fetched your branch. I updated some files in my local dir then I tried to push (first time) by using "git push --set-upstream sim3-GA-interface" (my remote dir that I want to push) but error shown as follow:

could you please show me how to resolve it?

@Jimmy-INL, personally, I think that function is to ensure if the user do not provide the sequence 'simulate', then it is set to 'simulate' by default.

[joshua-cogliati-inl]

Thank you for your patience, @joshua-cogliati-inl . I successfully fetched your branch. I updated some files in my local dir then I tried to push (first time) by using "git push --set-upstream sim3-GA-interface" (my remote dir that I want to push) but error shown as follow: could you please show me how to resolve it?

The remote name is what should come after --set-upstream. This is probably origin for you. (You can see all the remote names with git remote)

It will probably create a new branch called sim3-GA-interface-work, and it is probably simplest to create a new pull request with that branch (it is possible to modify your original branch, but that require reseting or rebasing, which are easy to mess up).

[Nhatkhang]

Thank you for your patience, @joshua-cogliati-inl . I successfully fetched your branch. I updated some files in my local dir then I tried to push (first time) by using "git push --set-upstream sim3-GA-interface" (my remote dir that I want to push) but error shown as follow: could you please show me how to resolve it?

The remote name is what should come after --set-upstream. This is probably origin for you. (You can see all the remote names with git remote)

It will probably create a new branch called sim3-GA-interface-work, and it is probably simplest to create a new pull request with that branch (it is possible to modify your original branch, but that require reseting or rebasing, which are easy to mess up).

Thank you, I made it to a new branch called sim3-GA-interface-work, thus I will probably close this PR and create a new one for this new branch.

[joshua-cogliati-inl]

Note: this has been replaced by #1992

[wangcj05]

This one can be closed.