Rw correlated arma

doc/user_manual/postprocessor.tex

@@ -125,7 +125,7 @@ \subsubsection{BasicStatistics}
       For example, if we define ``sen'' as the prefix for \textbf{sensitivity}, target ``y'' and feature ``x'', then
       variable ``sen\_y\_x'' will be defined by RAVEN.
       \nb These variable will be used by RAVEN for the internal calculations. It is also accessible by the user through
-      \textbf{DataObjects} and \textbf{OutStreams}. 
+      \textbf{DataObjects} and \textbf{OutStreams}.
   \end{itemize}
    %
   \nb If the weights are present in the system then weighted quantities are calculated automatically. In addition, if a matrix quantity is requested (e.g. Covariance matrix, etc.), only the weights in the output space are going to be used for both input and output space (the computation of the joint probability between input and output spaces is not implemented yet).
@@ -172,7 +172,7 @@ \subsubsection{BasicStatistics}
 \end{lstlisting}
 
 In this case, the RAVEN variables ``mean\_x01, mean\_x02, sen\_x01\_a, sen\_x02\_a, sen\_x01\_b, sen\_x02\_b''
-will be created and accessible for the RAVEN entities \textbf{DataObjects} and \textbf{OutStreams}. 
+will be created and accessible for the RAVEN entities \textbf{DataObjects} and \textbf{OutStreams}.
 
 \textbf{Example (Static, multiple matrix nodes):} This example shows how multiple nodes can specify
 particular metrics multiple times to include different target/feature combinations.  This postprocessor
@@ -1097,7 +1097,7 @@ \subsubsection{Interfaced}
 In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are available:
 
 \begin{itemize}
-   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, ID of the temporal variable. Default is ``time''. 
+   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, ID of the temporal variable. Default is ``time''.
    \nb Used just in case the  \xmlNode{pivotValue}-based operation  is requested
     \item \xmlNode{operator}, \xmlDesc{string, optional field}, the operation to perform on the output space:
       \begin{itemize}
@@ -1115,10 +1115,10 @@ \subsubsection{Interfaced}
         \item \textbf{celing}, find the value that is the nearest with respect to the \xmlNode{pivotValue} but greater then the  \xmlNode{pivotValue}
         \item \textbf{interpolate}, if the exact  \xmlNode{pivotValue}  can not be found, interpolate using a linear approach
        \end{itemize}
-       
+
        \nb Valid just in case \xmlNode{pivotValue} is present
      \item \xmlNode{row}, \xmlDesc{int, optional field}, the row index at which the outputs need to be extracted.
-       \nb This node can be inputted only if \xmlNode{operator} and \xmlNode{pivotValue} are not present   
+       \nb This node can be inputted only if \xmlNode{operator} and \xmlNode{pivotValue} are not present
 \end{itemize}
 
 This example will show how the XML input block would look like:
@@ -1699,15 +1699,15 @@ \subsubsection{RavenOutput}
 \subsubsection{ETImporter}
 \label{ETImporterPP}
 The \textbf{ETImporter} post-processor has been designed to import Event-Tree (ET) object into
-RAVEN. This is performed by saving the structure of the ET (from file) as a \textbf{PointSet} (only \textbf{PointSet} are allowed). 
+RAVEN. This is performed by saving the structure of the ET (from file) as a \textbf{PointSet} (only \textbf{PointSet} are allowed).
 Since an ET is a static Boolean logic structure, the \textbf{PointSet} is structured as follows:
 \begin{itemize}
   \item Input variables of the \textbf{PointSet} are the branching conditions of the ET. The value of each input variable can be:
   \begin{itemize}
     \item  0: event did occur (typically upper branch)
     \item  1: event did not occur (typically lower branch)
     \item -1: event is not queried (no branching occured)
-  \end{itemize}   
+  \end{itemize}
   \item Output variables of the \textbf{PointSet} are the ID of each branch of the ET (i.e., positive integers greater than 0)
 \end{itemize}
 Since several ET file formats are available, as of now only the OpenPSA format (see https://open-psa.github.io/joomla1.5/index.php.html) is supported.
@@ -1750,7 +1750,7 @@ \subsubsection{ETImporter}
     ...
     <PostProcessor name="ETImporter" subType="ETImporter">
       <fileFormat>OpenPSA</fileFormat>
-    </PostProcessor>   
+    </PostProcessor>
     ...
   </Models>
  ...

doc/user_manual/rom.tex

@@ -36,12 +36,12 @@ \subsection{ROM}
 These sub-types are specified in the \xmlAttr{subType} attribute and should be
 one of the following:
 \begin{itemize}
-  \item \xmlString{GaussPolynomialRom}, for both static and time-dependent regression 
-  \item \xmlString{HDMRRom}, for both static and time-dependent regression 
-  \item \xmlString{NDinvDistWeight}, for both static and time-dependent regression 
-  \item \xmlString{NDSpline}, for both static and time-dependent regression 
+  \item \xmlString{GaussPolynomialRom}, for both static and time-dependent regression
+  \item \xmlString{HDMRRom}, for both static and time-dependent regression
+  \item \xmlString{NDinvDistWeight}, for both static and time-dependent regression
+  \item \xmlString{NDSpline}, for both static and time-dependent regression
   \item \xmlString{SciKitLearn}, for both static and time-dependent regression and classification
-  \item \xmlString{MSR}, for both static and time-dependent regression 
+  \item \xmlString{MSR}, for both static and time-dependent regression
   \item \xmlString{ARMA}, for time-dependent stochastic regression (time series generator)
   \item \xmlString{PolyExponential}, for time-dependent regression
   \item \xmlString{DMD}, for time-dependent regression
@@ -2814,7 +2814,7 @@ \subsubsection{SciKitLearn}
 It has been more than 70 years since Warren McCulloch and Water Pitts modeled the first
 artificial neural network (ANN) that mimicked the way brains work. These days, deep learning
 based on ANN is showing outstanding results for solving a wide variety of robotic tasks in
-the areas of perception, planning, localization, and control. 
+the areas of perception, planning, localization, and control.
 %
 \textbf{Multi-layer Perceptron (MLP)} is a supervised learning algorithm that can learn
 a non-linear function approximator for either classifcation or regression. It is different
@@ -2908,26 +2908,40 @@ \subsubsection{SciKitLearn}
 %%%% ROM Model - ARMA  %%%%%%%
 \subsubsection{ARMA}
 \label{subsubsec:arma}
-The ARMA sub-type contains a single ROM type, based on an autoregressive moving average time series model.
+The ARMA sub-type contains a single ROM type, based on an autoregressive moving average time series model with
+Fourier signal processing, sometimes referred to as a FARMA.
 %
 ARMA is a type of time dependent model that characterizes the autocorrelation between time series data. The mathematic description of ARMA is given as
 \begin{equation*}
 x_t = \sum_{i=1}^p\phi_ix_{t-i}+\alpha_t+\sum_{j=1}^q\theta_j\alpha_{t-j},
 \end{equation*}
-where $x$ is a vector of dimension $n$, and $\phi_i$ and $\theta_j$ are both $n$ by $n$ matrices. When $q=0$, the above is 
-autoregressive (AR); when $p=0$, the above is moving average (MA). The user is allowed to provide upper and lower limits for $p$ 
-and $q$ (see below), and the training process will choose the optimal $p$ and $q$ that fall into the user-specified range. When 
-training ARMA, the input needs to be a synchronized HistorySet. For unsynchronized data, use PostProcessor methods to 
-synchronize the data before training ARMA.
-
-The ARMA model implemented allows an option to use Fourier series to detrend the time series before fitting to ARMA model to 
-train. The Fourier trend will be stored in the trained ARMA model for data generation. The following equation describes the detrend 
+where $x$ is a vector of dimension $n$, and $\phi_i$ and $\theta_j$ are both $n$ by $n$ matrices. When $q=0$, the above is
+autoregressive (AR); when $p=0$, the above is moving average (MA).
+While plans for optimizing $p$ and $q$ are under consideration, currently setting $Pmin$ and $Pmax$ to the
+same value is required, and similarly for $Qmin$ and $Qmax$.
+%The user is allowed to provide upper and lower limits for $p$
+%and $q$ (see below), and the training process will choose the optimal $p$ and $q$ that fall into the user-specified range.
+When
+training an ARMA, the input needs to be a synchronized HistorySet. For unsynchronized data, use PostProcessor methods to
+synchronize the data before training an ARMA.
+
+The ARMA model implemented allows an option to use Fourier series to detrend the time series before fitting to ARMA model to
+train. The Fourier trend will be stored in the trained ARMA model for data generation. The following equation
+describes the detrending
 process.
 \begin{equation*}
 x_t = y_t - \sum_m\left\{\sum_{k=1}^{K_m}a_k\sin(2\pi kf_mt)+\sum_{k=1}^{K_m}b_k\cos(2\pi kf_mt)\right\},
 \end{equation*}
 where $K_m$ and $f_m$ are user-defined parameters.
 
+By default, each target in the training will be considered independent and have an unique ARMA for each
+target.  Correlated targets can be specified through the \xmlNode{correlate} node, at which point
+the correlated targets will be trained together using a vector ARMA (or VARMA). Due to limitations in
+the VARMA, in order to seed samples the VARMA must be trained with the node \xmlNode{seed}, which acts
+independently from the global random seed used by other RAVEN entities.
+
+Both the ARMA and VARMA make use of the \texttt{statsmodels} python package.
+
 %
 In order to use this Reduced Order Model, the \xmlNode{ROM} attribute
 \xmlAttr{subType} needs to be \xmlString{ARMA} (see the example
@@ -2936,12 +2950,20 @@ \subsubsection{ARMA}
 \subnodeIntro
 
 \begin{itemize}
-  \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, defines the pivot variable (e.g., time) that is non-decreasing in 
+  \item \xmlNode{pivotParameter}, \xmlDesc{string, required field}, defines the pivot variable (e.g., time) that is non-decreasing in
   the input HistorySet.
-  \default{Time}
-  \item \xmlNode{Features}, \xmlDesc{comma separated string, required field}, defines the features (e.g., scaling). Note that only 
+  \item \xmlNode{Features}, \xmlDesc{comma separated string, required field}, defines the features (e.g., scaling). Note that only
   one feature is allowed for \xmlString{ARMA} and in current implementation this is used for evaluation only.
-  \item \xmlNode{Target}, \xmlDesc{comma separated string, required field}, defines the variables of the time series.
+  \item \xmlNode{Target}, \xmlDesc{comma separated string, required field}, defines the variable(s) of the
+    time series.  Should include the pivot parameter (or Index).
+  \item \xmlNode{correlate}, \xmlDesc{comma separated string, optional field}, indicates the listed variables
+    should be considered as influencing each other, and trained together instead of independently.  This node
+    can only be listed once, so all variables that are desired for correlation should be included.  \nb The
+    correlated VARMA takes notably longer to train than the independent ARMAs for the same number of targets.
+  \item \xmlNode{seed}, \xmlDesc{integer, optional field}, provides seed for ONLY the VARMA sampling.  Has no
+    effect on ARMA single-target sampling or other RAVEN entities.  Must be provided before training; it cannot
+    be changed once trained.
+  \default{True}
   \item \xmlNode{reseedCopies}, \xmlDesc{boolean, optional field}, if True then whenever the ARMA is copied, a
     random reseeding will be performed to ensure different histories.
   \default{True}
@@ -2995,12 +3017,12 @@ \subsubsection{PolyExponential}
 \begin{itemize}
   \item $\mathbf{z}$ is the independent  monotonic variable (e.g. time)
   \item $\mathbf{X}$  is the vector of the other independent (parametric) variables  (Features)
-  \item $\mathbf{P_{i}}(X)$ is a polynomial of rank M function of the parametric space X 
-  \item  $\mathbf{Q_{i}}(X)$ is a polynomial of rank M function of the parametric space X 
-  \item  $\mathbf{N}$ is the number of requested exponential terms. 
+  \item $\mathbf{P_{i}}(X)$ is a polynomial of rank M function of the parametric space X
+  \item  $\mathbf{Q_{i}}(X)$ is a polynomial of rank M function of the parametric space X
+  \item  $\mathbf{N}$ is the number of requested exponential terms.
 \end{itemize}
-It is crucial to notice that this model is quite suitable for FOMs whose drivers are characterized by an exponential-like behavior. 
-In addition, it is important to notice that the exponential terms' coefficients are computed running a genetic-algorithm optimization 
+It is crucial to notice that this model is quite suitable for FOMs whose drivers are characterized by an exponential-like behavior.
+In addition, it is important to notice that the exponential terms' coefficients are computed running a genetic-algorithm optimization
 problem, which is quite slow in case of increasing number of ``numberExpTerms''.
 %
 In order to use this Reduced Order Model, the \xmlNode{ROM} attribute
@@ -3010,23 +3032,23 @@ \subsubsection{PolyExponential}
 \subnodeIntro
 
 \begin{itemize}
-  \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, defines the pivot variable (e.g., time) that represents the 
+  \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, defines the pivot variable (e.g., time) that represents the
   independent monotonic variable
   \default{time}
   \item \xmlNode{Features}, \xmlDesc{comma separated string, required field}, defines the features (i.e. input parameters) of this
   model
   \item \xmlNode{Target}, \xmlDesc{comma separated string, required field}, defines output FOMs that are going to be predicted
   \item \xmlNode{numberExpTerms}, \xmlDesc{integer, optional field}, the number of exponential terms to be used ($N$ above)
    \default{3}
-  \item \xmlNode{coeffRegressor}, \xmlDesc{string, optional field}, defines which regressor to use for interpolating the 
+  \item \xmlNode{coeffRegressor}, \xmlDesc{string, optional field}, defines which regressor to use for interpolating the
    exponential coefficient. Available are ``spline'',``poly'' and ``nearest''.
     \default{spline}
-  \item \xmlNode{polyOrder}, \xmlDesc{integer, optional field}, the polynomial order to be used for interpolating the exponential 
-  coefficients. Only valid in case of  \xmlNode{coeffRegressor} set to ``poly''. 
+  \item \xmlNode{polyOrder}, \xmlDesc{integer, optional field}, the polynomial order to be used for interpolating the exponential
+  coefficients. Only valid in case of  \xmlNode{coeffRegressor} set to ``poly''.
    \default{2}
   \item \xmlNode{tol}, \xmlDesc{float, optional field}, relative tolerance of the optimization problem (differential evolution optimizer)
    \default{1e-3}
-  \item \xmlNode{maxNumberIter}, \xmlDesc{integer, optional field}, maximum number of iterations (generations) for the 
+  \item \xmlNode{maxNumberIter}, \xmlDesc{integer, optional field}, maximum number of iterations (generations) for the
   optimization problem  (differential evolution optimizer)
    \default{5000}
 \end{itemize}
@@ -3050,7 +3072,7 @@ \subsubsection{PolyExponential}
   ...
 </Simulation>
 \end{lstlisting}
-Once the ROM is trained (\textbf{Step} \xmlNode{RomTrainer}), its coefficients can be exported into an XML file 
+Once the ROM is trained (\textbf{Step} \xmlNode{RomTrainer}), its coefficients can be exported into an XML file
 via an \xmlNode{OutStream} of type \xmlAttr{Print}. The following variable/parameters can be exported (i.e. \xmlNode{what} node
 in \xmlNode{OutStream} of type \xmlAttr{Print}):
 \begin{itemize}
@@ -3072,9 +3094,9 @@ \subsubsection{PolyExponential}
     <Print name = 'dumpAllCoefficients'>
       <type>xml</type>
       <source>PolyExp</source>
-      <!--  
+      <!--
         here the <what> node is omitted. All the available params/coefficients
-        are going to be printed out 
+        are going to be printed out
       -->
     </Print>
     <Print name = 'dumpSomeCoefficients'>
@@ -3110,31 +3132,31 @@ \subsubsection{DMD}
      \item \textit{hodmd}, for high order DMD.
    \end{itemize}
    \default{dmd}
-  \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, defines the pivot variable (e.g., time) that represents the 
+  \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, defines the pivot variable (e.g., time) that represents the
   independent monotonic variable
   \default{time}
   \item \xmlNode{Features}, \xmlDesc{comma separated string, required field}, defines the features (i.e. input parameters) of this
   model
   \item \xmlNode{Target}, \xmlDesc{comma separated string, required field}, defines output FOMs that are going to be predicted
-  \item \xmlNode{rankSVD}, \xmlDesc{integer, optional field}, defines the truncation rank to be used for the SVD. 
+  \item \xmlNode{rankSVD}, \xmlDesc{integer, optional field}, defines the truncation rank to be used for the SVD.
      Available options are:
      \begin{itemize}
-     \item \textit{-1}, no truncation is performed 
+     \item \textit{-1}, no truncation is performed
      \item \textit{0}, optimal rank is internally computed
      \item \textit{>1}, this rank is going to be used for the truncation
    \end{itemize}
    \default{-1}
-  \item \xmlNode{energyRankSVD}, \xmlDesc{float, optional field},  energy level ($0.0 < float < 1.0$) used to compute the rank such 
-    as computed rank is the number of the biggest singular values needed to reach the energy identified by  
+  \item \xmlNode{energyRankSVD}, \xmlDesc{float, optional field},  energy level ($0.0 < float < 1.0$) used to compute the rank such
+    as computed rank is the number of the biggest singular values needed to reach the energy identified by
     \xmlNode{energyRankSVD}. This node has always priority over  \xmlNode{rankSVD}
     \default{None}
-  \item \xmlNode{rankTLSQ}, \xmlDesc{integer, optional field}, $int > 0$ that defines the truncation rank to be used for the total 
+  \item \xmlNode{rankTLSQ}, \xmlDesc{integer, optional field}, $int > 0$ that defines the truncation rank to be used for the total
    least square problem. If not inputted, no truncation is applied
    \default{None}
-   \item \xmlNode{exactModes}, \xmlDesc{bool, optional field}, True if the exact modes need to be computed (eigenvalues and 
+   \item \xmlNode{exactModes}, \xmlDesc{bool, optional field}, True if the exact modes need to be computed (eigenvalues and
    eigenvectors),   otherwise the projected ones (using the left-singular matrix after SVD).
   \default{True}
-  \item \xmlNode{optimized}, \xmlDesc{float, optional field}, True if the amplitudes need to be computed minimizing the error 
+  \item \xmlNode{optimized}, \xmlDesc{float, optional field}, True if the amplitudes need to be computed minimizing the error
    between the modes and all the time-steps or False, if only the 1st timestep only needs to be considered
    \default{True}
 
@@ -3161,7 +3183,7 @@ \subsubsection{DMD}
   ...
 </Simulation>
 \end{lstlisting}
-Once the ROM  is trained (\textbf{Step} \xmlNode{RomTrainer}), its parameters/coefficients can be exported into an XML file 
+Once the ROM  is trained (\textbf{Step} \xmlNode{RomTrainer}), its parameters/coefficients can be exported into an XML file
 via an \xmlNode{OutStream} of type \xmlAttr{Print}. The following variable/parameters can be exported (i.e. \xmlNode{what} node
 in \xmlNode{OutStream} of type \xmlAttr{Print}):
 \begin{itemize}
@@ -3189,9 +3211,9 @@ \subsubsection{DMD}
     <Print name = 'dumpAllCoefficients'>
       <type>xml</type>
       <source>DMD</source>
-      <!--  
+      <!--
         here the <what> node is omitted. All the available params/coefficients
-        are going to be printed out 
+        are going to be printed out
       -->
     </Print>
     <Print name = 'dumpSomeCoefficients'>
@@ -3203,4 +3225,4 @@ \subsubsection{DMD}
   </OutStreams>
   ...
 </Simulation>
-\end{lstlisting}
+\end{lstlisting}