Skip to content

Commit

Permalink
Logical Model (#1232)
Browse files Browse the repository at this point in the history 
New Logical model entity. Addition of a test to show how to use it.
  • Loading branch information
@wangcj05
wangcj05 authored May 28, 2020
1 parent 041c7ae commit 8849fe3
Show file tree
Hide file tree
Showing 33 changed files with 1,240 additions and 56 deletions.
177 changes: 177 additions & 0 deletions doc/user_manual/logical_model.tex
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%% Logical Model %%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%

\subsection{LogicalModel}
\label{subsec:models_LogicalModel}
The \textbf{LogicalModel} is a model aimed to execute ROMs,
Codes and ExternalModels via a user provided control function. Basically, the control function
utilizes the inputs generated by RAVEN and the control logic provided by the user to determine which
model to execute.
\nb For this type of model, we currently require all models listed under \textbf{LogicalModel} should
have the same inputs and outputs from RAVEN point of view.

The specifications of a LogicalModel must be defined within the XML block
\xmlNode{LogicalModel}.
%
This XML node needs to contain the attributes:

\vspace{-5mm}
\begin{itemize}
\itemsep0em
\item \xmlAttr{name}, \xmlDesc{required string attribute}, user-defined name
of this LogicalModel.
%
\nb As with the other objects, this is the name that can be used to refer to
this specific entity from other input blocks in the XML.
\item \xmlAttr{subType}, \xmlDesc{required string attribute}, must be kept
empty.
%
\end{itemize}
\vspace{-5mm}

Within the \xmlNode{LogicalModel} XML node, the multiple entities that constitute
this LogicalModel needs to be inputted.

\begin{itemize}
\item \xmlNode{Model}, \xmlDesc{XML node, required parameter}.
%
The text portion of this node needs to contain the name of the Model
%
\assemblerAttrDescription{Model}
%
\nb The user can provided various \xmlNode{Model} entities, including \xmlString{Code},
\xmlString{ROM} and \xmlString{ExternalModel}.

\item \xmlNode{ControlFunction}, \xmlDesc{XML node, required parameter}.
%
The text portion of this node needs to contain the name of the function.
%
\assemblerAttrDescription{ControlFunction}
\nb In order to work properly, this function must have a method named ``evaluate"
that returns a single python str object representing the model that would be
executed.
\end{itemize}

\textbf{Example (LogicalModel using external models):}
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
...
<Models>
<ExternalModel ModuleToLoad="sum" name="sum" subType="">
<variables>x, y, z</variables>
</ExternalModel>

<ExternalModel ModuleToLoad="minus" name="minus" subType="">
<variables>x, y, z</variables>
</ExternalModel>

<ExternalModel ModuleToLoad="multiply" name="multiply" subType="">
<variables>x, y, z</variables>
</ExternalModel>

<LogicalModel name="logical" subType="">
<Model class="Models" type="ExternalModel">sum</Model>
<Model class="Models" type="ExternalModel">minus</Model>
<Model class="Models" type="ExternalModel">multiply</Model>
<ControlFunction class="Functions" type="External">control</ControlFunction>
</LogicalModel>
</Models>
...
<Steps>
<MultiRun name="mc">
<Input class="DataObjects" type="PointSet">inputHolder</Input>
<Model class="Models" type="LogicalModel">logical</Model>
<Sampler class="Samplers" type="MonteCarlo">MonteCarlo</Sampler>
<Output class="DataObjects" type="PointSet">outSet</Output>
<Output class="DataObjects" type="PointSet">tagetSet</Output>
<Output class="OutStreams" type="Print">dumpOut</Output>
</MultiRun>
</Steps>
...
</Simulation>

\end{lstlisting}

Corresponding Python function for \xmlNode{ControlFunction}:
\begin{lstlisting}[language=python]
def evaluate(self):
"""
Method required by RAVEN to run this as an external model.
@ In, self, object, object to store members on
@ Out, model, str, the name of external model that
will be executed by hybrid model
"""
model = None
if self.x > 0 and self.y >1:
model = 'sum'
elif self.x > 0 and self.y <= 1:
model = 'multiply'
else:
model = 'minus'
return model
\end{lstlisting}

\textbf{Example (LogicalModel using codes):}
\begin{lstlisting}[style=XML,morekeywords={subType,debug,repeat,name,class,type}]
<Simulation>
...
<Models>
<Code name="poly" subType="GenericCode">
<executable>logicalCode/poly_code.py</executable>
<clargs arg="python" type="prepend"/>
<clargs arg="-i" extension=".one" type="input"/>
<fileargs arg="aux" extension=".two" type="input"/>
<fileargs arg="output" type="output"/>
</Code>
<Code name="exp" subType="GenericCode">
<executable>logicalCode/exp_code.py</executable>
<clargs arg="python" type="prepend"/>
<clargs arg="-i" extension=".one" type="input"/>
<fileargs arg="aux" extension=".two" type="input"/>
<fileargs arg="output" type="output"/>
</Code>
<LogicalModel name="logical" subType="">
<Model class="Models" type="Code">poly</Model>
<Model class="Models" type="Code">exp</Model>
<ControlFunction class="Functions" type="External">control</ControlFunction>
</LogicalModel>
</Models>
...
<Steps>
<MultiRun name="logicalModelCode">
<Input class="Files" type="">gen.one</Input>
<Input class="Files" type="">gen.two</Input>
<Model class="Models" type="LogicalModel">logical</Model>
<Sampler class="Samplers" type="Stratified">LHS</Sampler>
<Output class="DataObjects" type="PointSet">samples</Output>
<Output class="OutStreams" type="Print">samples</Output>
</MultiRun>
</Steps>
...
</Simulation>
\end{lstlisting}

Corresponding Python function for \xmlNode{ControlFunction}:
\begin{lstlisting}[language=python]
def evaluate(self):
"""
Method required by RAVEN to run this as an external model.
@ In, self, object, object to store members on
@ Out, model, str, the name of external model that
will be executed by hybrid model
"""
model = None
if self.x > 0.5 and self.y > 1.5:
model = 'poly'
else:
model = 'exp'

return model
\end{lstlisting}
%
\nb For these examples, the user needs to provide all the inputs for the models listed
under \textbf{LogicalModel}, i.e. Files for the \textbf{Code} and DataObject for
the \textbf{ExternalModel} defined in the \textbf{LogicalModel}.
47 changes: 26 additions & 21 deletions doc/user_manual/model.tex
View file
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ \section{Models}

\newcommand{\nIterNoChangeDescriptionA}[1]
{
\xmlNode{n\_iter\_no\_change}, \xmlDesc{integer, optional field}, number of iterations with no improvement to wait before early stopping.
\xmlNode{n\_iter\_no\_change}, \xmlDesc{integer, optional field}, number of iterations with no improvement to wait before early stopping.
%
\ifthenelse{\equal{#1}{}}{}{\default{#1}}
}
Expand Down Expand Up @@ -800,7 +800,7 @@ \section{Models}
}
\newcommand{\warmStartMLPDescription}[1]
{
\xmlNode{warm\_start}, \xmlDesc{bool, optional field}, when set to True, reuse the solution of
\xmlNode{warm\_start}, \xmlDesc{bool, optional field}, when set to True, reuse the solution of
previous call to fit as initialization, otherise, just erase the previous solution.
%
\ifthenelse{\equal{#1}{}}{}{\default{#1}}
Expand Down Expand Up @@ -979,8 +979,8 @@ \subsection{Code}
\item \xmlNode{executable} \xmlDesc{string, required field} specifies the path
of the executable to be used.

\item \xmlNode{walltime} \xmlDesc{string, optional field} specifies the maximum
allowed run time of the code; if the code running time is greater than the specified
\item \xmlNode{walltime} \xmlDesc{string, optional field} specifies the maximum
allowed run time of the code; if the code running time is greater than the specified
walltime then the code run is stopped. The stopped run is then considered as if it crashed.
%
\nb Both absolute and relative path can be used. In addition, the relative path
Expand All @@ -1001,7 +1001,7 @@ \subsection{Code}
\begin{itemize}
\item \xmlAttr{type} \xmlDesc{string, required field} specifies the type of
command-line argument to add. Options include \xmlString{input},
\xmlString{output}, \xmlString{prepend}, \xmlString{postpend},
\xmlString{output}, \xmlString{prepend}, \xmlString{postpend},
\xmlString{text}, and \xmlString{python}.
%
\item \xmlAttr{arg} \xmlDesc{string, optional field} specifies the flag to
Expand Down Expand Up @@ -1546,19 +1546,19 @@ \subsection{EnsembleModel}
\nb All the inputs here specified need to be listed in the Steps where the EnsembleModel
is used.
\item \xmlNode{Output}, \xmlDesc{string, optional field},
represents the output entities that need to be linked to this sub-model. \nb The \xmlNode{Output}s here
specified are not part
represents the output entities that need to be linked to this sub-model. \nb The \xmlNode{Output}s here
specified are not part
of the determination of the EnsembleModel execution but represent an additional storage of results from the
sub-models. For example, if the \xmlNode{TargetEvaluation} is of type PointSet (since just scalar data needs to
be transferred to other
sub-models. For example, if the \xmlNode{TargetEvaluation} is of type PointSet (since just scalar data needs to
be transferred to other
models) and the sub-model is able to also output history-type data, this Output can be of type HistorySet.
Note that the structure of each Output dataObject must include only variables (either input or output) that are
Note that the structure of each Output dataObject must include only variables (either input or output) that are
defined among the model.
As an example, the Output dataObjects cannot contained variables that are defined at the Ensemble model
As an example, the Output dataObjects cannot contained variables that are defined at the Ensemble model
level.
%
The user can specify as many \xmlNode{Output} (s) as needed. The optional \xmlNode{Output}s can be of
both classes ``DataObjects'' and ``Databases''
The user can specify as many \xmlNode{Output} (s) as needed. The optional \xmlNode{Output}s can be of
both classes ``DataObjects'' and ``Databases''
(e.g. \textit{PointSet}, \textit{HistorySet}, \textit{DataSet}, \textit{HDF5})
\nb \textbf{The \xmlNode{Output} (s) here specified MUST be listed in the Step in which the EnsembleModel is used.}
\end{itemize}
Expand All @@ -1585,7 +1585,7 @@ \subsection{EnsembleModel}
\xmlNode{varName} contains the value of the initial conditions (scalar or arrays, depending of the
type of variable). If an array needs to be inputted, the user can specify the attribute \xmlAttr{repeat}
and the code is going to repeat for \xmlAttr{repeat}-times the value inputted in the body.
\item \xmlNode{initialStartModels}, \xmlDesc{XML node, only required parameter when Picard's iteration is
\item \xmlNode{initialStartModels}, \xmlDesc{XML node, only required parameter when Picard's iteration is
activated},
specifies the list of models that will be initially executed. \nb Do not input this node for non-Picard calculations,
otherwise an error will be raised.
Expand All @@ -1596,13 +1596,13 @@ \subsection{EnsembleModel}
\newline \xmlNode{TargetEvaluation} determines how the data are going to be transferred from a model to
the other. If for example the chain of models is $A \rightarrow B$:}}
\begin{itemize}
\item \textcolor{red} { \textbf{ If model $B$ expects as input scalars and outputs time-series, the \xmlNode{TargetEvaluation}
of the model $B$ will be a \textit{HistorySet} and the \xmlNode{TargetEvaluation} of the model $A$ will be either
\item \textcolor{red} { \textbf{ If model $B$ expects as input scalars and outputs time-series, the \xmlNode{TargetEvaluation}
of the model $B$ will be a \textit{HistorySet} and the \xmlNode{TargetEvaluation} of the model $A$ will be either
a \textit{PointSet} or a \textit{DataSet} (where the output variables that need to be transferred to the model $A$ are scalars) } }
\item \textcolor{red} { \textbf{ If model $B$ expects as input scalars and time-series and outputs time-series or scalars or both, the \xmlNode{TargetEvaluation}
of the model $B$ will be a \textit{DataSet} and the \newline \xmlNode{TargetEvaluation} of the model $A$ will be either
\item \textcolor{red} { \textbf{ If model $B$ expects as input scalars and time-series and outputs time-series or scalars or both, the \xmlNode{TargetEvaluation}
of the model $B$ will be a \textit{DataSet} and the \newline \xmlNode{TargetEvaluation} of the model $A$ will be either
a \textit{HistorySet} or a \textit{DataSet} } }
\item \textcolor{red} { \textbf{ If both model $A$ and $B$ expect as input scalars and output scalars, the \xmlNode{TargetEvaluation}
\item \textcolor{red} { \textbf{ If both model $A$ and $B$ expect as input scalars and output scalars, the \xmlNode{TargetEvaluation}
of the both models $A$ and $B$ will be \textit{PointSet}s } }
\end{itemize}

Expand Down Expand Up @@ -1727,7 +1727,7 @@ \subsection{HybridModel}
\vspace{-5mm}

Within the \xmlNode{HybridModel} XML node, the multiple entities that constitute
this HybridModel needs to be inputted.
this HybridModel needs to be inputted.

\begin{itemize}
\item \xmlNode{Model}, \xmlDesc{XML node, required parameter}.
Expand All @@ -1751,7 +1751,7 @@ \subsection{HybridModel}
``CrossValidation``.
%
\assemblerAttrDescription{Model}
%
%
\item \xmlNode{TargetEvaluation}, \xmlDesc{XML node, required parameter}.
%
The text portion of this node needs to contain the name of a data object defined in the \xmlNode{DataObjects} block.
Expand Down Expand Up @@ -1928,3 +1928,8 @@ \subsection{HybridModel}
%
\nb For this example, the user needs to provide all the inputs for the \textbf{HybridModel}, i.e. Files for the
\textbf{Code} and DataObject for the \textbf{ROM} defined in the \textbf{HybridModel}.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Logical Model %%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{logical_model.tex}
3 changes: 2 additions & 1 deletion framework/Models/Factory.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@
from .Code import Code
from .EnsembleModel import EnsembleModel
from .PostProcessor import PostProcessor
from .HybridModels import HybridModel
from .HybridModels import HybridModel
from .HybridModels import LogicalModel

__base = 'Model'
__interFaceDict = {}
Expand Down
14 changes: 3 additions & 11 deletions framework/Models/HybridModels/HybridModel.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -502,21 +502,13 @@ def submit(self,myInput,samplerType,jobHandler,**kwargs):
self.modelIndicator[prefix] = 1
else:
self.modelIndicator[prefix] = 0
## Ensemble models need access to the job handler, so let's stuff it in our
## catch all kwargs where evaluateSample can pick it up, not great, but
## will suffice until we can better redesign this whole process.
kwargs['jobHandler'] = jobHandler
self.raiseADebug("Submit job with job identifier: {}, Runing ROM: {} ".format(kwargs['prefix'], self.romValid))
kwargs['useROM'] = self.romValid
## This may look a little weird, but due to how the parallel python library
## works, we are unable to pass a member function as a job because the
## pp library loses track of what self is, so instead we call it from the
## class and pass self in as the first parameter
jobHandler.addClientJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, prefix, kwargs)
self.raiseADebug("Submit job with job identifier: {}, Runing ROM: {} ".format(kwargs['prefix'], self.romValid))
HybridModelBase.submit(self,myInput,samplerType,jobHandler,**kwargs)

def _externalRun(self,inRun, jobHandler):
"""
Method that performs the actual run of the essembled model (separated from run method for parallelization purposes)
Method that performs the actual run of the hybrid model (separated from run method for parallelization purposes)
@ In, inRun, tuple, tuple of Inputs (inRun[0] actual input, inRun[1] type of sampler,
inRun[2] dictionary that contains information coming from sampler)
@ In, jobHandler, instance, instance of jobHandler
Expand Down
22 changes: 22 additions & 0 deletions framework/Models/HybridModels/HybridModelBase.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,28 @@ def createNewInput(self,myInput,samplerType,**kwargs):
@ Out, newInputs, dict, dict that returns the new inputs for each sub-model
"""

def submit(self,myInput,samplerType,jobHandler,**kwargs):
"""
This will submit an individual sample to be evaluated by this model to a
specified jobHandler as a client job. Note, some parameters are needed
by createNewInput and thus descriptions are copied from there.
@ In, myInput, list, the inputs (list) to start from to generate the new
one
@ In, samplerType, string, is the type of sampler that is calling to
generate a new input
@ In, jobHandler, JobHandler instance, the global job handler instance
@ In, **kwargs, dict, is a dictionary that contains the information
coming from the sampler, a mandatory key is the sampledVars' that
contains a dictionary {'name variable':value}
@ Out, None
"""
## Hybrid models need access to the job handler, so let's stuff it in our
## catch all kwargs where evaluateSample can pick it up, not great, but
## will suffice until we can better redesign this whole process.
prefix = kwargs['prefix']
kwargs['jobHandler'] = jobHandler
jobHandler.addClientJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, prefix, kwargs)

@Parallel()
def evaluateSample(self, myInput, samplerType, kwargs):
"""
Expand Down
Loading

0 comments on commit 8849fe3

Please sign in to comment.