Enabling profiling in RAVEN

[PaulTalbot-INL]

---

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

Closes #1317

What are the significant changes in functionality due to this change request?

Extends the options in raven_framework to include a --profile mode, which enables line_profiler during the run and activates the @profile decorator to indicate which methods should be profiled.

If not in --profile mode, then the @profile decorator is ignored (passthrough).

One method (Simulation.setInputFiles) was added to the profiling list just for demonstration and testing.

Note that the line_profiler library was added as an optional one, so running --profile without this installed will crash with something like "bash: kernprof: command not found".

The results of profiling is a line-by-line analysis of tagged methods with many factors, including:

• Number of times the line is called
• Total time taken on that line
• Time per call taken on that line
• Percentage of method time taken on that line

For example, profiling Driver.raven() as:

  @profile
  def raven():
    """
      A worker function that allows the computation of the main RAVEN execution
      to be offloaded to another thread, freeing the main thread for UI
      interaction (Qt requires UI to be handled on the main thread of execution)
    """
    simulation.initialize()
    simulation.run()

    ## If there is an associated UI application, then we can quit it now that
    ## we are done, the main thread does not know when this done presumably
    ## because this thread still is technically running as long as the app,
    ## which both threads can see, has not called quit. Otherwise, we could do
    ## this after the while loop below.
    if simulation.app is not None:
      simulation.app.quit()``

then running a test case, like

~/projects/raven/raven_framework --profile test_restart_MC.xml

prints, at the end of a RAVEN run ...

Wrote profile results to Driver.py.lprof
Timer unit: 1e-06 s

Total time: 0.480699 s
File: /Users/talbpw/projects/raven/framework/Driver.py
Function: raven at line 259

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   259                                             @profile
   260                                             def raven():
   261                                               """
   262                                                 A worker function that allows the computation of the main RAVEN execution
   263                                                 to be offloaded to another thread, freeing the main thread for UI
   264                                                 interaction (Qt requires UI to be handled on the main thread of execution)
   265                                               """
   266         1        458.0    458.0      0.1      simulation.initialize()
   267         1     480239.0 480239.0     99.9      simulation.run()
   268
   269                                               ## If there is an associated UI application, then we can quit it now that
   270                                               ## we are done, the main thread does not know when this done presumably
   271                                               ## because this thread still is technically running as long as the app,
   272                                               ## which both threads can see, has not called quit. Otherwise, we could do
   273                                               ## this after the while loop below.
   274         1          2.0      2.0      0.0      if simulation.app is not None:
   275                                                 simulation.app.quit()

---

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

• 1. Review all computer code.
• 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
• 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
• 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
• 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
• 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
• 7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
• 8. If an analytic test is changed/added is the the analytic documentation updated/added?
• 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

[PaulTalbot-INL]

On my mac machine, I saaw 636 passed, 99 skipped, with line_profiler installed (but not pillow).

[alfoa]

One comment. In addition, can you create a "mini"-developer guide for how to use it (basically your PR description with some additions here and there"?

[alfoa]

since this is a decorator, can we add this in RAVEN decorator module?

[PaulTalbot-INL]

I suppose we could; it would just mean adding that import to any file we want to use profiling on. I was able to mask it in the decorators just fine.

[PaulTalbot-INL]

I added a section to the Python Optimization wiki page.

[moosebuild]

Job Test qsubs on cefcb67 : invalidated by @alfoa

hpc issue?

[alfoa]

Checklist passed.

PR can be merged.

[PaulTalbot-INL]

I added this to the RAVEN 2.1 tracking list.