From eb95597a6205feb90610396636834ad0338c7249 Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Thu, 11 May 2017 14:46:08 -0600 Subject: [PATCH 1/9] user guide for single value ravenoutput plots generated --- doc/user_manual/model.tex | 30 +-- doc/user_manual/postprocessor.tex | 56 +++-- .../StatisticalAnalysis/comparingSamplers.xml | 233 ++++++++++++++++++ .../comparingSamplers/meanPlotter_scatter.pdf | Bin 0 -> 11430 bytes .../comparingSamplers/referenceInput.xml | 39 +++ .../comparingSamplers/varPlotter_scatter.pdf | Bin 0 -> 11892 bytes 6 files changed, 318 insertions(+), 40 deletions(-) create mode 100644 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml create mode 100644 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter.pdf create mode 100755 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/referenceInput.xml create mode 100644 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter.pdf diff --git a/doc/user_manual/model.tex b/doc/user_manual/model.tex index 7b55835973..e869e2ffbb 100644 --- a/doc/user_manual/model.tex +++ b/doc/user_manual/model.tex @@ -29,13 +29,13 @@ \section{Models} These aliases can be used anywhere in the RAVEN input to refer to the #1 variables. % - In the body of this node the user specifies the name of the variable that the model is going to use + In the body of this node the user specifies the name of the variable that the model is going to use (during its execution). % The actual alias, usable throughout the RAVEN input, is instead defined in the - \xmlAttr{variable} attribute of this tag. + \xmlAttr{variable} attribute of this tag. \\The user can specify aliases for both the input and the output space. As sanity check, RAVEN - requires an additional required attribute \xmlAttr{type}. This attribute can be either ``input'' or ``output''. + requires an additional required attribute \xmlAttr{type}. This attribute can be either ``input'' or ``output''. % \nb The user can specify as many aliases as needed. % @@ -900,7 +900,7 @@ \subsection{Dummy} ... - + another_name_for_this_variable_in_the_model @@ -1236,18 +1236,6 @@ \subsubsection{Method: \texttt{def run}} \input{postprocessor.tex} -\paragraph{Method: dataObjectLabelFilter} -The \xmlNode{HSPS} Post-Processor performs a filtering of the dataObject. This particular filtering is based on the labels generated by any clustering algorithm. -Given the selected label, this Post-Processor filters out all histories or points having a different label. -In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required: - -\begin{itemize} - \item \xmlNode{dataType}, \xmlDesc{string, required field}, type of dataObject (HistorySet or PointSet) - \item \xmlNode{label}, \xmlDesc{string, required field}, varaiable which contains the cluster labels - \item \xmlNode{clusterIDs}, \xmlDesc{int, required field}, cluster labels considered -\end{itemize} - - % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%% EnsembleModel Model %%%%%% @@ -1339,16 +1327,16 @@ \subsection{EnsembleModel} % \begin{itemize} \item \xmlNode{maxIterations}, \xmlDesc{integer, optional field}, - maximum number of Picard's iteration to be performed (in case the iteration scheme does + maximum number of Picard's iteration to be performed (in case the iteration scheme does not previously converge). \default{30}; \item \xmlNode{tolerance}, \xmlDesc{float, optional field}, - convergence criterion. It represents the L2 norm residue below which the Picard's iterative scheme is + convergence criterion. It represents the L2 norm residue below which the Picard's iterative scheme is considered converged. \default{0.001}; \item \xmlNode{initialConditions}, \xmlDesc{XML node, required parameter (if Picard's activated)}, Within this sub-node, the initial conditions for the input variables (that are part of a loop) need to be specified in sub-nodes named with the variable name (e.g. \xmlNode{varName}). The body of the - \xmlNode{varName} contains the value of the initial conditions (scalar or arrays, depending of the - type of variable). If an array needs to be inputted, the user can specify the attribute \xmlAttr{repeat} + \xmlNode{varName} contains the value of the initial conditions (scalar or arrays, depending of the + type of variable). If an array needs to be inputted, the user can specify the attribute \xmlAttr{repeat} and the code is going to repeat for \xmlAttr{repeat}-times the value inputted in the body. \end{itemize} \end{itemize} @@ -1403,7 +1391,7 @@ \subsection{EnsembleModel} 45.0 - + thermalConductivityComputation diff --git a/doc/user_manual/postprocessor.tex b/doc/user_manual/postprocessor.tex index 82213c0fe8..4292cda90e 100644 --- a/doc/user_manual/postprocessor.tex +++ b/doc/user_manual/postprocessor.tex @@ -101,6 +101,7 @@ \subsubsection{BasicStatistics} the \textbf{variationCoefficient} will be \textbf{INF}. \item \textbf{skewness}: skewness \item \textbf{kurtosis}: excess kurtosis (also known as Fisher's kurtosis) + \item \textbf{samples}: the number of samples in the data set used to determine the statistics. \end{itemize} The matrix quantities available for request are: \begin{itemize} @@ -110,7 +111,6 @@ \subsubsection{BasicStatistics} \item \textbf{NormalizedSensitivity}: matrix of normalized sensitivity coefficients. \nb{It is the matrix of normalized VarianceDependentSensitivity} \item \textbf{VarianceDependentSensitivity}: matrix of sensitivity coefficients dependent on the variance of the variables - \item \textbf{samples}: the number of samples in the data set used to determine the statistics. \end{itemize} If all the quantities need to be computed, this can be done through the \xmlNode{all} node, which requires the \xmlNode{targets} and \xmlNode{features} sub-nodes. @@ -1282,10 +1282,10 @@ \subsubsection{Interfaced} \paragraph{Method: dataObjectLabelFilter} This Post-Processor allows to filter the portion of a dataObject, either PointSet or HistorySet, with a given clustering label. -A clustering algorithm associates a unique cluster label to each element of the dataObject (PointSet or HistorySet). -This cluster label is a natural number ranging from $0$ (or $1$ depending on the algorithm) to $N$ where $N$ is the number of obtained clusters. -Recall that some clustering algorithms (e.g., K-Means) receive $N$ as input while others (e.g., Mean-Shift) determine $N$ after clustering has been performed. -Thus, this Post-Processor is naturally employed after a data-mining clustering techniques has been performed on a dataObject so that each clusters +A clustering algorithm associates a unique cluster label to each element of the dataObject (PointSet or HistorySet). +This cluster label is a natural number ranging from $0$ (or $1$ depending on the algorithm) to $N$ where $N$ is the number of obtained clusters. +Recall that some clustering algorithms (e.g., K-Means) receive $N$ as input while others (e.g., Mean-Shift) determine $N$ after clustering has been performed. +Thus, this Post-Processor is naturally employed after a data-mining clustering techniques has been performed on a dataObject so that each clusters can be analyzed separately. In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required, @@ -1301,10 +1301,10 @@ \subsubsection{Interfaced} The user is required to provide the following information: \begin{itemize} - \item the set of input variables. For each variable the following need to be specified: + \item the set of input variables. For each variable the following need to be specified: \begin{itemize} \item the set of values that imply a reliability value equal to $1$ for the input variable - \item the set of values that imply a reliability value equal to $0$ for the input variable + \item the set of values that imply a reliability value equal to $0$ for the input variable \end{itemize} \item the output target variable. For this variable it is needed to specify the values of the output target variable that defines the desired outcome. \end{itemize} @@ -1314,11 +1314,11 @@ \subsubsection{Interfaced} \item $R_0$ Probability of the outcome of the output target variable (nominal value) \item $R^{+}_i$ Probability of the outcome of the output target variable if reliability of the input variable is equal to $0$ \item $R^{-}_i$ Probability of the outcome of the output target variable if reliability of the input variable is equal to $1$ -\end{itemize} +\end{itemize} Available measures are: \begin{itemize} - \item Risk Achievement Worth (RAW): $RAW = R^{+}_i / R_0 $ + \item Risk Achievement Worth (RAW): $RAW = R^{+}_i / R_0 $ \item Risk Achievement Worth (RRW): $RRW = R_0 / R^{-}_i$ \item Fussell-Vesely (FV): $FV = (R_0 - R^{-}_i) / R_0$ \item Birnbaum (B): $B = R^{+}_i - R^{-}_i$ @@ -1339,7 +1339,7 @@ \subsubsection{Interfaced} \end{itemize} \textbf{Example:} -This example shows an example where it is desired to calculate all available risk importance measures for two input variables (i.e., pumpTime and valveTime) +This example shows an example where it is desired to calculate all available risk importance measures for two input variables (i.e., pumpTime and valveTime) given an output target variable (i.e., Tmax). A value of the input variable pumpTime in the interval $[0,240]$ implies a reliability value of the input variable pumpTime equal to $0$. A value of the input variable valveTime in the interval $[0,60]$ implies a reliability value of the input variable valveTime equal to $0$. @@ -1356,14 +1356,14 @@ \subsubsection{Interfaced} pumpTime valveTime Tmax - + ... ... \end{lstlisting} -This Post-Processor allows the user to consider also multiple datasets (a data set for each initiating event) and calculate the global risk importance measures. +This Post-Processor allows the user to consider also multiple datasets (a data set for each initiating event) and calculate the global risk importance measures. This can be performed by: \begin{itemize} \item Including all datasets in the step @@ -1399,7 +1399,7 @@ \subsubsection{Interfaced} outcome outRun1 outRun2 - + ... ... @@ -1409,8 +1409,8 @@ \subsubsection{Interfaced} \end{itemize} This post-processor can be made time dependendent if a single HistorySet is provided among the other data objects. -The HistorySet contains the temporal profiles of a subset of the input variables. This temporal profile can be only -boolean, i.e., 0 (component offline) or 1 (component online). +The HistorySet contains the temporal profiles of a subset of the input variables. This temporal profile can be only +boolean, i.e., 0 (component offline) or 1 (component online). Note that the provided history set must contains a single History; multiple Histories are not allowed. When this post-processor is in a dynamic configuration (i.e., time-dependent), the user is required to specify an xml node \xmlNode{temporalID} that indicates the ID of the temporal variable. @@ -1431,12 +1431,12 @@ \subsubsection{Interfaced} outcome outRun1 time - + ... ... - ... + ... outRun1 timeDepProfiles @@ -1486,6 +1486,11 @@ \subsubsection{RavenOutput} file. This will appear as an entry in the output \xmlNode{DataObject} and the corresponding column are the values extracted from this file. If not specified, RAVEN will attempt to find a suitable integer ID to use, and a warning will be raised. + + When defining the \xmlNode{DataObject} that this postprocessor will write to, and when using the static + (non-\xmlNode{dynamic}) form of the postprocessor, the \xmlNode{input} space should be given as + \xmlString{ID}, and the output variables should be the outputs specified in the postprocessor. See the + examples below. In the data object, the variable values will be keyed on the \xmlString{ID} parameter. \end{itemize} Each value that needs to be extracted from the file needs to be specified by one of the following \xmlNode{output} nodes within the \xmlNode{File} node: @@ -1526,7 +1531,11 @@ \subsubsection{RavenOutput} \end{lstlisting} -The RAVEN input to extract this information would appear as follows: + +The RAVEN input to extract this information would appear as follows. +We include an example of defining the \xmlNode{DataObject} that this postprocessor will write out to, for +further clarity. + \begin{lstlisting}[style=XML] ... @@ -1537,7 +1546,7 @@ \subsubsection{RavenOutput} ... ... - + ans|val1 ans|val2 @@ -1550,6 +1559,15 @@ \subsubsection{RavenOutput} ... ... + + ... + + ID + first,second + + ... + + ... \end{lstlisting} diff --git a/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml b/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml new file mode 100644 index 0000000000..9ff94042ec --- /dev/null +++ b/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml @@ -0,0 +1,233 @@ + + + + + sampleMC, sampleGrid, sampleLHS, + statsMC, statsGrid, statsLHS, + readStats, plot + + comparingSamplers + 8 + + + framework/user_guide/StatisticalAnalysis/comparingStatistics + talbpw + 2017-05-11 + PostProcessors.BasicStatistics,PostProcessors.RavenOutput + + This test is an example for the user guide to draw from. + It compares the basic statistics from three sampling strategies in a plot. + + + + + referenceInput.xml + output_stat_analysis_mc.xml + output_stat_analysis_grid.xml + output_stat_analysis_lhs.xml + + + + + ../physicalCode/analyticalbateman/AnalyticalDplMain.py + + + + + + A,B,C,D + A,B,C,D + A,B,C,D + + + + C|expectedValue + C|variance + + + C|expectedValue + C|variance + + + C|expectedValue + C|variance + + + + + + + 0 + 1000 + + + 0.00000001 + 0.0000001 + + + + + + + 1296 + 42 + + + sigma + + + decayConstant + + + sigma + + + decayConstant + + + + + sigma + 0 1 + + + decayConstant + 0 1 + + + sigma + 0 1 + + + decayConstant + 0 1 + + + + + 42 + + + sigma + 0 1 + + + decayConstant + 0 1 + + + sigma + 0 1 + + + decayConstant + 0 1 + + + + + + + referenceInput.xml + bateman + mc + samplesMC + + + referenceInput.xml + bateman + grid + samplesGrid + + + referenceInput.xml + bateman + lhs + samplesLHS + + + samplesMC + statsPP + statsfileMC + + + samplesGrid + statsPP + statsfileGrid + + + samplesLHS + statsPP + statsfileLHS + + + statsfileMC + statsfileGrid + statsfileLHS + statsReader + all_stats + + + all_stats + meanPlotter + varPlotter + + + + + + sigma-A,sigma-B,decay-A,decay-B + A,B,C,D + + + sigma-A,sigma-B,decay-A,decay-B + A,B,C,D + + + sigma-A,sigma-B,decay-A,decay-B + A,B,C,D + + + ID + mean,variance + + + + + + + screen,pdf + + <text>Means</text> + + + + + scatter + all_stats|Input|ID + all_stats|Output|mean + + Sampler Number + Value + + + + + screen,pdf + + <text>Variances</text> + + + + + scatter + all_stats|Input|ID + all_stats|Output|variance + + Sampler Number + Value + + + + + diff --git a/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter.pdf b/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter.pdf new file mode 100644 index 0000000000000000000000000000000000000000..2a74b3907ab1310bdc6aab19ec406a39011ab5e8 GIT binary patch literal 11430 zcmb_?2{@Er)Hp>L%1%o4B3qW(hq3S3_pML{gULR|mJ}6QC<#TfL_(V-g|ykSChgKD zN_JAwBHw)nMSQ={^MC#`&wcKF&wKB==bm%!Ip@CTiks``$)V+O2=Pm=po^6VG=zfu zTy`N;R3M~vm_HdpYLjRrZ$A$RX-@JW2SOOo!4gtaLy&#l;Esy(2H2aq>>|6;AO(6K zFn~rQQ+**k{D?rBQ~g}6$TY|SpwiI;=*gip2x;I0&}z-r+OxF*_#DJ`Go?~Vf1(f0))e2U>Y@jef?;G z01n!gF@p$3_s7&t4Hw$41?NHotk>rAtHx498 z#A)-J#6ProvO=6Btj?$U_~&q2bLCH^%&xB#`(4v{ET4*p&1=GQP6}oeo*3QN_h_X` zu|ZNkH1` za;w}gYEhb+cXc(jx)HL%$7Gm)>2V$m*70Cw<74LbU>1ts!nH1RJ>sQmLW!{^iv8x+ zROOdBEtU=SOh?vx`K1>}-BJszVH38=%J(tOTKjBog`MS!O}vm&t?HE@Z*LACQqTN3 z^14`hovCOJZlmGZ7DNAU(h}@HLYkd-ls(FNE+p#1GkAzHl5!!W@*zuUp4n|3_450D zsl50*J|jY-f%mcda5BFKkN!#yPtSA+xwkKH>cg4*SkA$(RSiFmPO1M|@xZ5U#52(4 zR~>dtA%Uy5r=FhNma}GouCB0N6Y!WhQ-Oxh7KYEVE60A_>MW_`# zd3iC2k9sKx=@>^1wL{6x%)HHvr7f~8QL~cl3Hs*St^eA0IO0ND2-nO}<|rJBB~dR%Bjv#~EL)XBmHXFsI0OXHe9~-BuUBPOxk-3l0uYZEY6%vY0N+ ze8c%)48Z^sJ6}3@^XApsXx5t^2!VXQ6;W;(wzq<&y?h=jlenj~>>H9;!wMQ7NcS13 zDH(HKm%mdi%6>UWe6T4E%8(h!PAw^JfAQYgf1SjRNC5@sPn7s|cv{!~Jo#j;s5zowFzV9NM@dT-6JTu*l5cvHsu@9WctMwLr_ zyCio7i?&nwF;7Zy_t&!W>TXmC)6MgLU-~TMDURofJG*HL;}ui&xHSyUqC2)l4qnkA z8a+C8KGU!2{sgB6gVnVqSTo-#{ugV2g3sp+7S@%>8a{1#hIL6R2A4n#JJaQC2n=rF zj*zn;Ca%pO;mgz!w^&%Mmt z%`q~90gbiHRzWWsQftie;@F2A+g3MJa6coj%{`M`A9qm9M8r}*yI-OJtC_%*bI~Fv zUbt>=U$fX*l8x9dj-QT3ozDY#AFRVE3tUtop5JPh-rQKnF2gca)p}Rgdfi18E*}3o zo!V3BgFOmw9#xdqeVN=5SW<=Jw;%ghwm!k6Y*GRFep0o2;QOzY`R5%3mS7<4{+Gi* zGy%InG$skkYidv&+98?y1zDl~?zS*X+OCm1wp5<11NT^(5~P9xS>4}diB!G0dsJ=% z`TmWpOJ?1Zk{>D$M?`)*Hx!7)_CQFQZ+nJ{B_Szg}GH0i!GFR+_V-x$?DqvdvJ zd(%WfKc7eVG=utpB6f+j!cow2)~dKr3ssNZ|TOAILccNc?$RilEB9o!{n`m$!lih{nmKC`sWR{NzYtC;f9`%_cBUoMJBy0 zUwU&*#&2=4GSyn5tQo8|~hYQJFG5+uQa;&MpBVi#^Nyrog1 zgC=X|9ji|(uI&2V8kU~+wC8sx>)@rVC;^@X0mX(|BMysrYTc%={bNc|cLqLCeyP2Q zezO09=+xaZ%KDM}X?TkwoQrge+P5hwk^5HNZ@E5I=<~X}VW^PMp`T@H`<|UKCjMHL zO9}iTWw4t|?#1hM!F#qPXg)f*E@iMMHb}0hlGWt&dM{BSVi^YCc0Z#Fld)Dz^QkOk zkXdfVAE%GqsO%vZme3ViD2Uv9EB-m=c)=+fX3TZAkIu@eY!{?Nx$b0bt~h+tvtR0% zdwp7GbWEb?#B0^`^e>fn(sh`3uyx5k<2fd@y9pzyyxK5p*9WH4q+8#j1JVeF>W9;dPVeYuvs_#zT+>RA36%vn+Pf5Mh z-^BO4*U=>PvQ*cm!k)wY&nKsUy=iMxPf&;`UP3g>4Fxc`h1zUlannZ!%`xzn&+#3@ zfLB-syQR}NKkY`=glZ@oDSbg);9I>BM;D6&=k1;oHro3(s-p+7vZ{?ePt-qFG+OiB z<}W;R+Fa13Ys!_+a8)4%aDX37qpKKLZKDr&UciQ`YBbMS zuMIKblfA;dEeJF=rDB%CbT-|#1$te)KNF)BNCanJ3sGRZ*g?L zOwuzp+PP=7l`Zcs_I?Xi&_;fKmtPqg!7*S|RQ>6Exa^)QTANzF7%PVuh}{~^p$)g|JF#&jlGX7Xoa+xQeazm(&aCHS}8@I_&vv&NR9EI=s5;DQarG6#wo zMFZy~#*WR>kUzz255+b=e8|;=A)qZ$I2?0{acg|G3o2e?n^A%$(R_;NNMSIu=0Pq& z&2nQJEDE<^^Q$H*C&Fep{Oo?ghPcU-J>zbY$rHba)G&>w?WC3z|kMS^X* zy{pSb`9sKyjMkRBWjaXSX6d$Unc7#n-%C>CD@&B>3Tsej9lqqJAB{0Nnby0-NazN8 zMorn_-js-1P6b>(OHjN~_3GD8J+~oR{ez~JZ*A=e!K(NNQpRxqx1xLdaT+tj+HuI{=7r$+nO8eN~$$?8T6J)s=Nr=Z6@z4*viG}R$)D9 zYB4zU5PN&jkWH26dYAohgmPMO-?}r3gL_Bxo;Af?@f6wHU?({s?)=J1LYK$Q^t!$8 z(We>1*;09dF@Zu*f5(v+%a1O07x!@}`^h3ZWE+xN*fDw+OfnQh8}`PheF;nM4V_~2 z9uHJr%y{gwgEs8KZKP{qKxGo)xH%#7C90rWW;DaJwP1YOOBQQjD&XverbI=_g&r1p zcDE%8S+U3D6K;+Tj_s^6NB zoUHU!3n%eHgj-b%2vIihK(6#<%MZ7u93tiWldqd9$0ufq@pbtniFNhy8wQ(rue6qRrWiA z(eJUfZGzF74-V#Pq<cG&Xir7mxgeoxd% z6I_?9x^|*VOIDJ_1gh-Vmw=KJTZ0}pcBB*KQagq(3>oj?w3kS=_#l0@7o#RwFz#p? zFOoN4A$Q3^mp8xz+7@ zUzza5v+*mAi1M}cZ24jz_4KUt`<61DvaiJ$>Z`sD2i~vly^?NSQf^VNRH($SnAh9% zqj30Fxop#Iu~-bV_cP)eg8zr>q&VK3R@;!+pE?N_P8Q_64^Hr&JaDr9cGFY-a#O<3 ze(|*|`}bzE8Gkm(3X0L_{wTeh%Jyw6;O4kS(Tys$TyNH%2Cld@nm0|a>Jxy2E@=##IF`4{&T4X-svcK911tbu4Ort(n*MYtz64 zr}+{0LrZAka%0zjln{%#0ZyPrI7Xfr0SUxM_f_y|IEBkrUeH*UrDa8Xia$a$V--VY z1V`BtlP))iL=(^pEVXWe5l;=op*@kVwRLpssy!LBO2d&mmqkoUBQF$dtO}82Dkx*- zZD&UM7*(>XIj}vb5B)^&#q;zq1x4ECc6%mSMpV>Ml4)a}gg_bg%e))h*X&Np5z69H z;(s%Au7{d8{NVdm|F1j0ZY>^8Z>_!S_F>II_pj2!d(LdIaC+^w?~$wTGp~L8-wKPa zT{{2aVCMrTY!=gSf$-22b3vfe!6jC@+;|j&B`jE}iMf%ljtEC*;1S1n2Af3~A80?d z>HXbVTw^`7C1&kDea^lTn~eijEqQ!gu;%gXXmxac5mV z@;s}n-mLNgC$?Vlx$LIn%4X}`qw8zqyU)rCIcEwm;vJG(qli9A3L2FO>uwa5zHu|> z5bf7$WjQIIexd!;A!0_pvtfutS;BfpwWrxD1Ge8nRJa~J=~pYfE&DF%p>e>oos7@& z8%wq|Kn5`)4@`fPW9hgWK}-VNeC z8Sy=Y;N8^Ex6-z8HWujX^FOXcnC3L_8NEoEOgobH=t=YF+fx%l`=%MO@Aoh+A(Z9D z%4qyQPClW68+@733AE8<<0)e8FTUR%GF>3}-SCO+)nQP*e29oEzDxu}=od^9##^+>;+unw#qOjeFW*PpVeq`z1$B zK22t-Tr2qGkad+kDMF)6@vMP4I^wEJR8D&EKGF2G@6?30rb>$ixcDLlCxjT>Z`+mE zcQe0cR0vyQmCKE2vEWP9{I4u{i<<#;D2@R5;cD4+^JS(tlw63%mPxb+fG<_Zl_t9Jx11P)tP{gzaR`<~lC;drezM zLvf<^X1>o6Wp@vWc}H?*n+=I^f$WIx#Bi#mnuN`f5e7XG;fO*zOqJk?j$JXW(?U)i zTM(;kwr$GP^Orbapg8W9gLrIN_V~`vSKZ?*uCBWymk`9VgZV|LX5y9C8XN-=2q!2q zxHY|(xnS?`+nA)C2<>aON79Z7BvZbVMQr3w%HJ6>@HxEx^F$-!FmIxr_{;MgC(OUA zL|fnSXs){bES(!NN=BSKnRs8E{Wr&{H}b;0u3qZL#X?kfJKMw^)nQ9v0ZEIhM8ca>RtawLR26 zBy8Xx7jJ$dGC1m`J_oT>;ATGZ$F`)?Y_y@HqQmFz@))$FyErNR;8df@=HDT7H(xzw z_(NaD%lKtTjsc}b^yHOOx_%0N5vt`HEV>lD1OcG?!vC z;YMV`HEa89+t)eTW6@H)oD6dBqL{mG3r~*;=HGi>6iuxOcU$3X_{3MEUb|gLf14&Z z>E{XlPr2q-y}L%FAiF1*qO)l(hgPRBNZ-D=`ta%ej;z>&$%%y<9*xE2jyl|PI!rQm zaOh+Tb$`g!@#aK4H+HC9C}116m%3_Sjs>R7LWtAEW6#)YjOBPcK>MiSb(6M7o0g>W z=8p!ad;P__GCDQc(O&|trkHt8oR+pdG0Lpm#JKX-)SJZkw(xhdagLpX#*(A$lncVV zmv3)$nL42EZFI)*zD%qxUw)Mo>)z*@ya9Xc>n7ZCbV9B%J`>}8>M72}!BhQU2=zI% zJ`DJeYahSpc;@JAd~%FiRjQ=!Lpp!agdBzSlJ^)}ZH zPeL+&{`kcx&yX#&gh2n#lZaK23yq$J)$^nJ{7D9a)7)%!b~fdTKx)n8C*{zJI7kkT zY0wx&MTm$Yg4E=EbJ>l-p{!5@yiLOgi~?arDfO*7)FB7DQYrp4KPrT#XJ;X#Dai+9E43)Jnd5>jnM$$(DONd4KOd4W0tshpO-M8< zg`P!~N1@=Qga4lv$e?PG0?G8O?*ER)j^al141~}#d0*ro_Q1JnEXebElBjT=8T5cd znLkO`8LWB~Y9I~n0-yWapYjKFN7phJUnR# zq%Q^ZRRH2N0UU%BK}Rb{5#Y3ihyX}}kS-7!unI{A6~ONeA%Pb7f^0CLz#mjN2niC# zfe19nJs^W1C&a8?{2?C59%)RPUH0seY%WJP|1bbc6i#{5b%!pk|6mTFl1BEi1ssNR z)gFO>Aqb?sK7CCP9(=N!r2=OAJXN8i{Xg`?gcM4bF%U5htWlffKZ{lYeC$9PfV&73 zS503JZ!$c&<_yiSmV-sQ(`Y@&Aeki&lhiKzuj?3dmLwB47y!A|3@P zfaD$qdM8Fe(f&wHUSE2xvSpg&x2q@5oYxq4D6p+_d0OMht zL`5Q~XJBwmUkp|O0oO1Q@S@;C0Gkyaiw6sz!2$Y$dFgE|4lqKIUhoP;dQDUS^a3~m zjR6asnHLA<#bW?tfMbK-;X$nkm<8ALf`R)H=uCiXcz(LjUQEO1VjQ6nk%p(%;*5D z2MY_G*FY~|otRhX`T*<1T%qgAtWLoCFu%Zb(VqbhSU2VhedQ=T;5(od)`2AoQ4z=w z)(hZa=9f90n5$u?0J$URHQb(8=(3)95BK>~V7-8)3fE45jKlmB?u;p>UrVt3W5xkD z>5c<7;%GP)2CwL5ot_~F4>N`z$Qryt(9Jw%=2Fy*F$dhBk1>HTbDj|bdO(L+L6}*$ zM0W=m`b8-OF!q?)K_tNOnXWDn{o^i}j||jr2Id1h9y%g#py@L$UkEIjPKiI*RWmSP zk7uW$Lcl7}M+8AYDe0{lpM;&AXU1mJBg^^My5?*se0fR_DB=v}zeaYx>(Io^+_BOG z7cS=9&!4+nNLLa7!(K=eJ=YCb?wPw~f3JAWb(?3$=fxa<=bb0MzrgtiWH|1S^P%_# z1wy!4wwZ%nJ;S%ZL+F72%nkktEfz#xXu!8w;unQpSa}e4p$G~@c(=$cc>3QT7Di`2 z322a)X4MTb?PA}xCow+Lbm)pRT&l?P`uhA%ezB=&LJ}Dg~3<|A)!lTh>U=Gm)ITS$> zg_4w}-xqWXas~I{qLEQ(y+WyEcLWMtKKlbc5DzwyJM;$zhmr6P@|}gDQ5aye7Q#?O z*pDuP0bBboOc6wQ|H6nMDYgg(TMYR3XI{`IEQVo$llr$G1_X-#zyNa>(? + + 300 + 1 1 1 + 1e14 1e14 1e14 + 0 100 200 400 + 100 100 100 + + + N1 + 1.0 + $RAVEN-decay-A|10$ + $RAVEN-sigma-A|10$ + 230 + + + N2 + 1.0 + $RAVEN-decay-B:0.000000006$ + $RAVEN-sigma-B:5$ + 200 + + + N3 + 1.0 + $RAVEN-decay-C:0.000000008$ + $RAVEN-sigma-C:3$ + 150 + + + N4 + 1.0 + $RAVEN-decay-D:0.000000009$ + $RAVEN-sigma-D:1$ + 100 + + + + diff --git a/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter.pdf b/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter.pdf new file mode 100644 index 0000000000000000000000000000000000000000..482cf3a61a9623d8c1628379dc787e9bffbfd035 GIT binary patch literal 11892 zcmb_?2{=_<)UYvJ%1lai%w)LpPIc@Bkg%_75fO^GO!LNZT@$oN(=Mw2KRlTs?> zO_4H{QZ&f7&qX2M`+5H7zn*QMv(DOUuf5jVYprwE7BkV*mPO0q5MuwlfXeC-Xb1&) zI~+hLD?><&U>|1)sX?NUJiT2ZqzTE@*$=`11T$#I4urFp6O5>^WPr7?!vSYU3M5a( z0Rt!$XR;TBhi?%`6SB9XxibZ_0i-mw0eR;j3WU@>2*|20^coAjE@THGwSp+S%qb*_ zGlZKbVD3h8a`$qD2=nB%y}c+9mU{h%9W)9678kIK?CtLZ!AsFFhme}i0q%~@X1l;- z;Ld_f^78W`k)6F9gBO=NkH2FFyjJjlGQU^^($Lw-ouuv^1Xzd3<`26{UX2G?w&wk5K`L{tXk9A z(c1~g-@w_+mEs1WVa-ql??)j!lMW(+vg1+>{T`t>qCb4+RcsVGc~f>vmK3FsDf%H> zwuX-eRyVIT`B$jr-q7l5PNsX;_UU$wb{>#bo7Q}=Wr)Jwi{+g{T-q}5YDDN3hsLUH_+D|@dWlzJT6X2?0rN)yyoy*riFxOP|l&yM#)mtCw)a@RGV z4GjC}681oSugi@DJ#vvPb}U-dIdLDqT6_Cs*KEK~a`pK>ztG+{Wuggv1TF6F42zp5 za2b)qVurmwq*Mj0>63{eQI%JF%PjZf@NzY6r*7Mvd7LS?Qyn8}_b;b(20>3pgV*&i ziZXlbI(+Xg0UNkNbI$GWgmFm6mli!{^LrJ)-f^-53@|Taf$A!WvsDe z`u&I}7ZB(7yu01*S#{0n;*=(1X-43=r;aK86Lt0U*AQ%h+`Ppj*KYm1T(vI81X7H@ z_EPZ3wr1|xDay|sIpSLKA&*l-ZylPIt9M7t>F`gfc^$Lp%9j4IyDQ>sINga?ZunO` zYCh`&ZXa{3I65w9trP#k9%p?xzK3f>Jt{oMuEZ<#03Cf@w`Z$V>QnrSqRsUGrWko3 zDBf4TJa?*@;?bYlH1SA<$L2yivED+QUm$b6%DsgC$ZM1mO<5dzzZ==4?%@N^_fo0v zz+m?4mRaeS2~TgdY|!pw(+nNu8u=jmj=9Fnwaic;r)+&^YPLg+T~lcN)&td@p$yO2 zZ(J;jt~E5Ny_YbT*Ho)xRiEv7bO2S&T(o^X@0pW5O-!1OPvR$Y1^8y_iOO-oG53Nv zY$99LZXWMwd_Vf7O~Lfe9?iz)%>`BBC=?g(nX7^nk96L`TrvBqjUBHv1=o8ZZ^ajN zz7CrENNMKoH@qEnCo;?{_spiTVgy6p$fMb}B&D26@+bZ+sz#S!Z?v#47W@)*zrY(y zUID%6tI)7xTFYj`5n&Jad!u0qRVFAC6b{9ds2#1EdL~Bwq!7Oj&7F>+qbzU3_pDRh zbY6k!lK64y^)$`z){C#g%u+A;U%OlZwQORo*lX(>P|TVkh4u_#;x6YTQ!0;ahV)+e z7}tkfRpi2$_miTJGvu(zM4L~p%gDFKU)45?J3E$N?zP`P;_V^xvMPPd{sKz z1*TTvYpvdgxSTJ;I5n;Yh?)Mwbzx%@%?ERoLl5Wnm-TG8(O^Uu!>4t03V%Q!HE|Wg z>tQ05o}AXO8)eJ-eBylDFO06i+2}_hwe-g-CVh`R)cx_*_Yh6l?4flp*&hlDTmQ`= zW~o2_FUetm=U=Kbyh;0d4K(vLR|H=%4_$;)j^z#iIgf)6lt`R&>eemE%)uqC_od$J z?@%;guaUcTMVR%Xzt~t?CX^%nAupriO83*(_CA}$_l5Jx+rM*<+k~eK94(YfQIEJQ z_A%qIf5^L=#sWbH1jqploEok6EC=fO46Un_&m|bqzMczfnvfmLOZ?oHv-#WR?D3CE zm0kl92Lgn<$-J1JO5DAT%-mXAm4mekeO^~S34Dy>>TzK;N~OJIv@>=CjlJ-`J>g@Q zG>Q5T<1d`^zJ70-U6sbXY8BBejn)4p8esNIB}295@FpG&Ihsw$bYrU!hGYC%wh49x zWxSa_8OXfvmbp*F{q$R*Ih2Y66Spo#N=TQ>s;bK&_dL`O>k;d_&@^bZZCdP#`i|4% znH^?#r}-**x@0f!>f)@GMKG9htIS{|c!hhqdfq7pkLN4Jskl)J83t87aWh#)C z+A7Me5~EW=z9DaUqCdeZ@t&)gxB$mX@hgT^4R2ngR7GZ_t24f}4u4Fktp6@(VEv{% za&kL~@9{Ci9DGTnOI5t1Q&Z4%qd`nmbQ70JR%i-KY1-*WFIYqhcNqqZH&)mXJ_u>% zf5XZo-@Yd)N_*VMQ55m8Je)Ivc%%H}jkl~szJ?RdXESBk4!Vh5I}{()@F2VL0;#z- zr}y(OCM}zg{8h97r#5TR0tTnBOuw?G#6)^E=JsLUL+XByK3r)(Gd=en(MEeLS_4v) z^)+cW=17(9|J1U(YT!f0#@m6n^Zsq~-^@qk;MIA!(ZaCU?)ugnOaX!}be{|GGRFi) z>32FJ6ZOLddE{tUzx9wWrQpfqkd1FCV zN^@+ah@p^~PTq)k305tEA-~KtKThy&*!y;ov!vZ32iSht>i6~fao^vBQ{pRACSKTO zmEGQYmsOf+_IhW7mc^zrWezT%TYVa{*<*w9{SWIZ?@rC^^Q*Xy;&FHtE^u-r!_*M}hoHnjSfb#fd^25hn;RH^X9(KCzwA^1?FEswBWtnj+;1_He ze0w{mf>Lfc%rrGzVE9?xG@%srfZ`JB>o&N-as&T9H@(Xz-*u^dV9FTb&s>EiI2B$C zOIYl3or0Z}Xrl)2tao&m4*cp2&dz*1_^Xe3?4R5SKCT2lg_b+|Y^HJKyW4_~PAW#+ z8Xa=~xuZX-=V+1eY(usC<`4HW@uuZC2dR!7UuPwS?wP;&m*ZWX4!4UFh71Xu_))#% z$dfDs;?H#j?tb4Tbq{gKK7F|<;P9RVwTDTYQpX05`^%QsGaIIF_7D~zR%7s8_i_d> zIU7aP9?L+w=dK#_#_C|VD!IxACkzCY@*~4;#PwoMm!$1x#MH37wO7huDUuZCxRonf z7jw;RL^9r`IkPV+I#GD~rAl`8RQ;`NO~!pJ12Rvz;sp-1VI-8+>*XF8Vn`?5_!{M# zNzl7${%G9p%H@Au#Yr#TmRFi#*5CK_L1uMuiiW&Ileo7YPoJLPjJEF}NY<~BrU*&V8*^A@s|MAup=YM=1 z>(m&N7vH3oWw9~PkVobcZzr~8+)}m9hFXg!a_O!WvF6@kE8E4O=D5^Mw)~e2(bC#a@(c1NSmX%l1cKa&wTWfA5zdTN3D^aE> zlX4mdtRI%qPeeZo#3d0{Y_6(Y8lyFIy#2ugjy4PdZHB_(7%L1qyEw~S_rQRI(oo>joX(DhYE2|}yHr=6^qm)4-zqZV>MIoQmNucz+DsK^ zpNck2${gOVFHp;x(^MTZoEmzET^?7=^kXAOAU}8@rwb;+6TcEJk|x zPxG7dzWpX-eE#@*z1N48C53E5P}+Q4gXZ0;@s+k0KkZZhx}kPcMWfmQ-Qh|J<){9K zjOD!eJ2}{$>MUlAOvlC_U~i7;v8Zs>3|LQuDrH`IzbQ*$EbN2!leXAPZbD%#RuZFP z_AkuEwYZ#&YOK9ZJW|9K5Dkur>VYc7~9lC%5sMXY*N7e~6r zE9I)jJ&YBejh9HI7(+QW6?AEFRt9{I0#lkpa(AcHN-LvsB0rr!yrJS-u@INt`RoXZ zbxirHO69{f=VGj)a{c|!)x_m(lG5mKqpppq4OSLufj5wL&vZfqkC%qIIE7T|bS9*vD1K4L zi9Z$MR1pC-FXMNtKr^t1$92+t12`Mh$KVYfXq&f zM2C*tWYcL>b^MfX#hG3H4_cpP6J;}=O%#nA9A>u`&oCX5Iy;QnAyM+#)+ltMa!dE% z`3t*$WY6}WN&O^T-Iw?Hy_J}0M%?3~gsaMPvF9d#(Z}cRp?;@Z{$>omT&>OK2+MO2 zET$H`1()M!xuW^R5pP}AA*7;hRd1HW)Tzty>#*Hvqpf5MuM*2XdcsB>rJ1ql`|Ikb*N-D{J z9gyHTb1bR(X4_-lt44$$BVrqwj)vv27<@9!^^aD4^H%B*ndR%G@3qgW<+ayY3Otzy zTR37js9iI?S8&POKuwwdT4jHpHqrP35qZ<2R6D>Y(dR^+5>dtfM&> z%2vI~q-)J)(FF9eAgh(2&(#F6X-sFU@A|lF-QgTcz21jg7ljNf!;7w{t_zf4D5+-T z?q)HEdXamJ%Z5YA`2x8d zioE^f=LgA!6ZgOE^7*p=%dRUE*`0S9oQ5_;x_ptEIGnY`)b6Fzk%x|6PdtwBel0Dp z`scz>WZ!)|Y%aq@iQxDwV~L+)ww+co-s}?9d?)VBSvdjwb9}UTo0QH7;z31u)%t`@wWXE)P9|)^ zBkG+@Npjgm-DxL?ImPyRf#TH(n{9VI&ZGC;djnDDcq+;Jj^LiW2GRoq-zWQNpA@%N z>}i2?qlNB==7%IB;)>gXrFk3KlkzyfL_f}5mr@sXzGNuz=a%PmD8`yt|MzTq5v4S@ z|J_~`dW%*x&dz)lp{+F|!~&1B^aF*g88KcI&DK7P2(zad{WO`g)u89c97T6GBV(y& zyYe8pNmDFLCZW&N<8`mLOK#)G%Vix=A9VU!6i;}xXxX-=Ho9uc45IHS-#PlBZ$@D2 zw|_cXE18;+&!z<|$4Xe*)tgUYtD3gQH*IgZVm~Te8WLn)&{QJ&^v$Wc9e4fskMi9V zace++8!i(Gx>=z0T*alY%7#7FeOC%k>alV6`|X4JqcT?a-h6|e_46vJAFv8Pj|yxK zmW#(6bk_7BxgINMzWep<)6*9i?A|py7bwtu%h=vXub0{3c^>-J@?NM2!%=wYy`Ds^ zDR1CNt8Hc7^|QJdn`y0+ojmtP8#BHHren%Fm4Z{bFBoaf2!zxytUvo^jjwhH?8=Z5@pbS*j79+= zyYn?C`I2?5pd+u&`g-5zydN|^2e)j3T+&ldBJ3<<_`=jCb9Y?~owl~HhPuZEb$w#v zOwNP{L_F7FBUbWVD@K0blbp^%89yaFalU~|w5nB{Kz_qXGpp*>Lv+k9Tk@ zAIEYfPlZZ`?bjEtmprp`!}`1<`?u!DX0P+6_3J)YbCRBa@)_x}pncTCyP1Qbs@u5_ z*?*twoeBQ+lU}eU++r2gtTj)?5|?{G6*GM=M(`oEOX{lXg^z~hoRa>Dmq%=}IxI>N zNctLdavEES*v^r}6Ig?=FJ!ydrk`$j9&mjtG$BVQoUtbFuaAC8?#^_$Vl3E-Y^k!a z&a-@(uQ3@V$<0n9`znHQ;HKc*B!BVk-ts7NQ-~9ty_y+|rVsmw z4CM5wv7)DZFQ*!NPNz#*p83eA)J98xW41prt}En~Oss9+n1RH{ZucTV?u$3KI?Nv1 z>8YP(dr$hf7EkeYN#?L#HE!R-)_13!@-+jiXrG91KXw!2VB>1MKaTno)Eo?4bk*A_ zO}Bi_tv&IaDv}jU45PM{y)?Pr#w#nl8sJJ%KqtzW=0^ zqsbFkMWO#^>TK@sK%th+wY|v)e;04T%iBUVeW65-K&sEzlV#BgI7k))(grj}K>;FS zh@h0bG#I|Y;85l$0)BVP5JsLrS4nP=5%Cj&SVd3St2sTQq1}%_{LLqNS!M=G&zjDD zj%0TqiZ>ZTQ>*C^(ui~rRF~D=Df4d!md<37IVg$Cnt2~2c_EN+b={CeA-hwn-f}1u zY|G&9_Z6sktCReksa5#@os5;c6UEIBLeJOfk-xv0%@%?zbHKTq`ANaEchkHLM8iuSc)Mjh^l3vWE@zd2FYiEtUUO-fz$=3N8BCN zyj(q<;mOtJd4}yAtaEW$i*JE85CMhy|IrZtuXM0z1qcVUh$EnYmK7iZmVhAQQ9$n~ z2m=ZQSWpRt8y@fpKZo(L;Pr;ugKZA*|EnPupJM?o++Z4u4Ua_wn}G!Yd;$Sj7lEh% z$%F5u1Yo~-Fu6S3@E8a~#3Mk12zUa72YH$T$YSJyIm;^m22kJ{?&0TH&_E$v9*l=+ z5*3J`pNGLcbun0Z1l+?+z>9(#0c=-zEFLU;o(8}L^HQ&|I3NfGYQxJDsXb91@C)Ds zGzKhieqJ1y7mopg0ig|kh6lX@5Ek518wSQBP=x^Z@cdMz!2$$40D)44g$HA(LRn-E z?x~gr&xC_-P_T$#w&A70J!n*Yz|+k)1b#thze&ws00!qtV8DvOFg(l^;2h=%HiJcV zE%rcVh($)I6maqY32beceb^3u(}%HOAk~-vCHVd~6__a$3PF9gK!8XfLW>PHgn1i) z?O=JM${N@OY!gcw)gEA*SZq{VS+EJ%9+ozkFX}y@0o%r6qplo<2XY6z!Zxr z!FB;+%+j`K6N^186rgtmwTG{lG^(!WpTju68*CS_R^i_6w{uwf?!Dl;sRudC<{dYX zCH1rw)n}vO#2-AM`gQ8r0dO<#`GKmzBLvl-W8gRrp1?jIXr6j)1Ys5;qd5eufC{5V z0L(&UvIMq1Pl5zt7AAEB@}5U^fT-Vc0a0gQeDg4OIPg#j9Rzkhf8_~*1p^oY#00QZ z>Qf(JI`db+FH`4$K#TxNqCWKpduaY@J~~koelULCcjM;g`r}L&eJOnGO&b{G{2YIt zFAX7T==sxeYJmP?82Wwc4(UjOsNZdnYM4bd5Z>kk^q<3di)c&y`I7A8&%8_2_y@T3 z6M+fX$_wIxlM&g2?q2?W5C|!-6#fjM;{0c!?st;0EC)jaDK0SoR}zMm0~r{KAWwvM zjqI|!|NUlpo^~(+4c;7?wE~R>SP!#~F%UhPZAJ8VOHJ?3vV4D9q|$nZevUS^|GazB zXSLP~^@sG2yXB;nS}Jx|)Zss0E+|b(GtbQ0W5=~|yiw+<8vBE^#9w=Vol`QrGo+?| ze1yg-c(BVSE&Oyh11IYnosGlIj!1@6B3IcPHH5D!sB_ix?iVgQOcUv9ed_OQ5SHMg zz%;!5LCOHssPFCsXGYZXl4kJn#d*EXAM*SoZQN3G(f|h!Dem50n((1aNlirz3N4Sq zqtR&K7SRM*6hQ)ol8~aF^mOuf1Sc6K50WT8p57EscL&JF$pvyCIeLJY4^XJ2KsLG* z11*P_!^%Kz6pD``68UGSoHyB3YKa8|k)2%-CLv1w+x0-P=FoB3K#+Qb1PtI;Ja7AKw9t@3=L|oEAh}+;QLpMMdM+2wtNmW zfw)pGC^Qk|3M=q1C{STo3B%$4hRLromiU*BP*^yjT0sv>0Qu2Mm^|?>UT|n|sCNY( z4)^!GIC+pmt-!+r%lZoj9}`%K2j_4rVFVECR>D?U8vz3c3;6fj4p0OF912#z6hJbx z0w#|}{k1lEJRC!pkClgG`*Ijq8OT3=!zg5uyQedmdOBR!%f%ZUgPxB}X5QYgSE0r$ ebN55wJ!_$?P9jsN+ZBx>f;SE^F)d?l#Qy-K#~?TW literal 0 HcmV?d00001 From 98c92b1175b01622c1e3369b330a0796f27cf566 Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Thu, 11 May 2017 14:46:43 -0600 Subject: [PATCH 2/9] associated guide docs --- doc/user_guide/statisticalAnalysisExample.tex | 1 + doc/user_guide/statisticsToDataObject.tex | 296 ++++++++++++++++++ 2 files changed, 297 insertions(+) create mode 100644 doc/user_guide/statisticsToDataObject.tex diff --git a/doc/user_guide/statisticalAnalysisExample.tex b/doc/user_guide/statisticalAnalysisExample.tex index 44d3320899..48865b43af 100644 --- a/doc/user_guide/statisticalAnalysisExample.tex +++ b/doc/user_guide/statisticalAnalysisExample.tex @@ -192,3 +192,4 @@ \section{Statistical Analysis through RAVEN} \end{table} \end{landscape} +\input{statisticsToDataObject.tex} diff --git a/doc/user_guide/statisticsToDataObject.tex b/doc/user_guide/statisticsToDataObject.tex new file mode 100644 index 0000000000..1d3321cad8 --- /dev/null +++ b/doc/user_guide/statisticsToDataObject.tex @@ -0,0 +1,296 @@ +\subsection{Plotting Multiple Statistical Analyses} +In section \ref{sec:SAraven} we covered how to perform statistical analysis and print the results to an XML +file. However, what if we want to use the statistics calculated in more RAVEN calculations, like plotting and +further postprocessing? + +In the following two sections, we consider two use cases. +\begin{itemize} + \item In the first case, we want to compare the statistics obtained by three different sampler types on the + same model. We want to run the sampling, analyze the statistics, then make two plots where we can see how + the means and variances of the data compare between the sampling strategies. The plots will have the + sampler type on the x-axis, and the metric values on the y-axis. + \item In the second case, we want to see the time evolution of the statistics; that is, the mean and 5/95 + percentile of the model output as it evolves in time. We want to plot this so that the x-axis is time, + and the y-axis is the model output value, with three series, one each for fifth percentile, mean, and + ninety-fifth percentile. +\end{itemize} + +Note that while we use the `BasicStatistics` postprocessor as an example, there are many other RAVEN +postprocessors that produce XML outputs that follow this same strategy. + +\subsubsection{Case 1: Comparing Samplers} +In this case, we are interested in comparing the statistics obtained by several samplers (Monte Carlo, Grid, +and Stratified/Latin Hypercube) on a plot. To do +this, we need to complete the following steps: +\begin{enumerate} + \item Sample the model using Monte Carlo, Grid, and Stratified samplers, storing results in separate point + sets. + \item Perform statistical analysis (mean and variance) on each of the point sets containing the samples, writing results to + separate RAVEN XML output files. + \item Read in the RAVEN XML output files, creating a point set with the sampler types as inputs and the mean + and variance as outputs. + \item Plot the metrics as a function of the sampler used. +\end{enumerate} + +These steps lead to the following \xmlNode{RunInfo} block, see especially the \xmlNode{Sequence}: +\xmlExample{framework/user_guide/StatisticalAnalysis/comparingSamplers.xml}{RunInfo} +Note that each sampling and statistics requires its own step, which grows our total steps from four +conceptually to eight in RAVEN. + +Next, let's define the \xmlNode{Steps} block that will execute this \xmlNode{sequence}: +\xmlExample{framework/user_guide/StatisticalAnalysis/comparingSamplers.xml}{Steps} + +The three ``sample'' steps take the code input (\xmlString{referenceInput}) as input, define the code itself +(\xmlString{bateman}), indicated the sampler to use in each, and then output the results to a +\xmlNode{PointSet}. + +The three ``stats'' steps take one of the point sets as input, use the basic statistics postprocessor as the +model, and output to a RAVEN XML output file. Note that we can re-use a single basic statistics +postprocessor, rather than create separate entities. + +In the \xmlString{readStats} step, the three RAVEN XML output files are inputs to the RavenOutput +postprocessor model, and result in a \xmlNode{PointSet} that has the statistics from each file in a single +data object. + +In the \xmlString{plot} step, the statistics data object is passed to two \xmlNode{OutStream} plotters. We +include the \xmlAttr{pauseAtEnd} attribute to ensure plots printed to screen are retained. + +With the \xmlNode{Steps} defined, we now define all the entities used throughout the calculation. We won't go +over them in great detail, but will point out a few notable features: +\begin{itemize} + \item Data Objects: + \begin{itemize} + \item There are three data objects for storing the samples from the samplers, and one to hold the + statistics. Note that while the output of the Bateman code is histories, since we only define + \xmlNode{PointSets} the code keeps only the final value for output values. + \end{itemize} + \item Files: + \begin{itemize} + \item In order for RAVEN to keep track of the XML files in which the statistics will be written, we + define them in the \xmlNode{Files} block. Note that these files need not exist before the RAVEN run + starts; they will be created in the run. + \item Additionally, we define the input template for the code itself, used in all the sampling steps. + \end{itemize} + \item Models: + \begin{itemize} + \item In the Models we define the interface to the Bateman code as well as the two postprocessors. + \item Note that in the \xmlNode{BasicStatistics} we request the expectedValue, variance, and number of samples as + metrics for all four outputs of the Bateman model, even though we only need the expectedValue and + variance for output C. This demonstrates the ability of the RavenOutput postprocessor to be selective + about the values it retains. + \item In the \xmlNode{RavenOutput} postprocessor, there are two operation modes, \xmlNode{dynamic} and + non-dynamic. Because the \xmlNode{dynamic} is not specified, we are operating in static mode (we look + at dynamic mode in the next section). This means we specify several files to load from. Ultimately + we want a point set that contains the following information: + \begin{center} + \begin{tabular}{c | c | c } + ID & mean & variance \\ \hline + 1 & MC mean & MC variance \\ + 2 & Grid mean & Grid variance \\ + 3 & LHS mean & LHS variance + \end{tabular} + \end{center} + To get these values, we assign a float file ID to each file in the \xmlAttr{ID} attribute. These + values can be whatever we want; they'll be used to keep track of which set of outputs come from which + sampling strategy. Within the \xmlNode{Files} node, we instruct the RAVEN XML output reader how to + read in values using the \xmlNode{output} nodes. The \xmlAttr{name} attribute determines what column + (or variable) in the PointSet a value will go in, and the text of the node gives a path in the XML to + find the value. For example, \xmlString{C|variance} instructs the postprocessor to look in the RAVEN + XML output file under the node \xmlNode{C} for a node named \xmlNode{expectedValue}, and use its + value. + + Over the process of reading all the files, entries for both the mean and variance are collected, + giving us a data set of realizations where the input space is the file ID and the output space are the + desired metrics. Note that it is not necessary to retain the original name of the statistics; we + chose the shorter name ``mean'' to read in the ``expectedValue'' entries. + \end{itemize} + \item Samplers and Distributions: + \begin{itemize} + \item The Distributions are unchanged from previous examples. + \item In defining the three samplers, we only note that we specified the number of samples so that they + were equivalent across all three samplers in an effort to make a fair comparison. + \end{itemize} + \item OutStreams: + \begin{itemize} + \item We define our two plots here, using fairly standard syntax. Note that when these plots are + generated, the File IDs are on the x-axis, while the statistics metric (mean or variance) values are + on the y-axis. The File IDs can be traced back to the IDs given in the definition of the + \xmlNode{RavenOutput} postprocessor. + \end{itemize} +\end{itemize} +When this code is run, the following plots are produced: +\begin{figure}[h!] + \centering + \includegraphics[width=0.7\linewidth]{../../tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter} + \caption{Sampler Comparison Plot, Means} +\end{figure} +\begin{figure}[h!] + \centering + \includegraphics[width=0.7\linewidth]{../../tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter} + \caption{Sampler Comparison Plot, Variance} +\end{figure} + + +%%%%%%%%%%%%%%%%%%% OLD +%In order to accomplish these tasks, the following RAVEN \textbf{Entities} need to be defined (these are +%similar to the statistics example in section \ref{sec:SAraven}): +%\begin{enumerate} +% \item \textbf{\textit{RunInfo}}: +% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{RunInfo} +% As shown in the other examples, the \textit{RunInfo} \textbf{Entity} is intended to set up the desired +% analysis . In this specific case, two steps (\xmlNode{Sequence}) are sequentially run +% using forty processors (\xmlNode{batchSize}). +% \\In the first step, the original physical model is sampled. The obtained results are analyzed with the +% Statistical Post-Processor. +% \item \textbf{\textit{Files}}: +% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Files} +% Since the driven code uses a single input file, in this section the original input is placed. As detailed in the user manual +% the attribute \xmlAttr{name} represents the alias that is going to be +% used in all the other input blocks in order to refer to this file. +% \\In addition, the output file of the \textit{PostProcess} \textbf{Step} is +% here defined (XML format). +% \item \textbf{\textit{Models}}: +% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Models} +% The goal of this example is to show how the +% principal statistical FOMs can be computed through RAVEN. +% \\Indeed, in addition to the previously explained Code +% model, a Post-Processor model (BasicStatistics) is here specified. +%Note that the post-process step is +%performed on all the variables with respect to the parameters used in this example ( $A,\, B,\, C \, and \, D$ +%with respect to $sigma-A,\,sigma-B,\, decay-A,$ and $decay-B$). +% \item \textbf{\textit{Distributions}}: +% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Distributions} +% In the Distributions XML section, the stochastic models for the +% uncertainties are reported. In +% this case 2 distributions are defined: +% \begin{itemize} +% \item $sigma \sim \mathbb{U}(0,1000)$, used to model the uncertainties +% associated with the Model \textit{sigma-A} and \textit{sigma-B} +% \item $decayConstant \sim \mathbb{U}(1e-8,1e-7)$, used to +% model the uncertainties +% associated with the Model \textit{decay-A} and \textit{decay-B}. +% \end{itemize} +% \item \textbf{\textit{Samplers}}: +% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Samplers} +% In order to obtained the data-set through which the statistical FOMs need to be computed, a \textit{MonteCarlo} sampling approach is here employed. +% \item \textbf{\textit{DataObjects}}: +% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{DataObjects} +% Int this block, two \textit{DataObjects} are defined: +% 1) PointSet named ``samplesMC'' used to collect the final outcomes of +% the code, +% 2) HistorySet named ``histories'' in which the full time responses of the +% variables $A,B,C,D$ are going to be stored. +% +% \item \textbf{\textit{Steps}}: +% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Steps} +% +% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% Finally, all the previously defined \textbf{Entities} can be combined in +% the \xmlNode{Steps} block. As inferable, +% 2 \xmlNode{Steps} have been inputted: +% \begin{itemize} +% \item \xmlNode{MultiRun} named ``sampleMC'', used to run the +% multiple +% instances of the driven code and +% collect the outputs in the two \textit{DataObjects}. As it can be +% seen, the \xmlNode{Sampler} is inputted to communicate to the +% \textit{Step} that the driven code needs to +% be perturbed through the Grid sampling strategy. +% \item \xmlNode{PostProcess} named ``statisticalAnalysisMC'', used +% compute all the statistical moments and FOMs based on the +% data obtained through the sampling strategy. As it can be noticed, +% the \xmlNode{Output} of the ``sampleMC'' \textit{Step} is the +% \xmlNode{Input} of the ``statisticalAnalysisMC'' \textit{Step}. +% \end{itemize} +%\end{enumerate} +% +%Tables \ref{ScalarMoments}-\ref{SensitivityComputed} show all the results of the \textit{PostProcess} +%step. +% +% +%\begin{landscape} +%\begin{table}[h!] +%\centering +%\caption{Computed Moments and Cumulants.} +%\label{ScalarMoments} +%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} +%\hline +%{\ul \textit{\textbf{Computed Quantities}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline +%\textit{expected value} & 5.97E-02 & 3.97E-01 & 9.82E-01 & 1.50E+00 & 5.57E-08 & 5.61E-08 & 5.07E+02 & 4.73E+02 \\ \hline +%\textit{median} & 2.45E-02 & 3.06E-01 & 9.89E-01 & 1.54E+00 & 5.73E-08 & 5.62E-08 & 5.11E+02 & 4.70E+02 \\ \hline +%\textit{variance} & 8.19E-03 & 6.00E-02 & 1.19E-02 & 1.49E-02 & 7.00E-16 & 6.83E-16 & 8.52E+04 & 8.64E+04 \\ \hline +%\textit{sigma} & 9.05E-02 & 2.45E-01 & 1.09E-01 & 1.22E-01 & 2.64E-08 & 2.61E-08 & 2.92E+02 & 2.94E+02 \\ \hline +%\textit{variation coefficient} & 1.52E+00 & 6.17E-01 & 1.11E-01 & 8.15E-02 & 4.75E-01 & 4.66E-01 & 5.75E-01 & 6.21E-01 \\ \hline +%\textit{skewness} & 2.91E+00 & 9.88E-01 & -1.49E-01 & -9.64E-01 & -6.25E-02 & -5.75E-02 & -2.18E-02 & 7.62E-02 \\ \hline +%\textit{kurtosis} & 9.56E+00 & -1.12E-01 & -6.98E-01 & -1.50E-01 & -1.24E+00 & -1.21E+00 & -1.21E+00 & -1.20E+00 \\ \hline +%\textit{percentile 5\%} & 2.87E-03 & 1.48E-01 & 7.89E-01 & 1.24E+00 & 1.42E-08 & 1.45E-08 & 5.08E+01 & 2.97E+01 \\ \hline +%\textit{percentile 95\%} & 2.51E-01 & 9.19E-01 & 1.16E+00 & 1.63E+00 & 9.54E-08 & 9.48E-08 & 9.59E+02 & 9.49E+02 \\ \hline +%\end{tabular} +%\end{table} +%\begin{table}[h!] +%\centering +%\caption{Covariance matrix.} +%\label{covarianceComputed} +%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} +%\hline +%{\ul \textit{\textbf{Covariance}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline +%\textbf{A} & 8.19E-03 & -1.11E-03 & -3.09E-03 & -1.13E-04 & -1.28E-09 & 5.14E-11 & -1.49E+01 & -3.74E-01 \\ \hline +%\textbf{B} & -1.11E-03 & 6.00E-02 & 2.26E-03 & -2.96E-02 & -7.80E-11 & -6.02E-09 & 7.00E+00 & -1.47E+00 \\ \hline +%\textbf{C} & -3.09E-03 & 2.26E-03 & 1.19E-02 & 7.15E-04 & -1.44E-09 & -4.11E-12 & 2.63E+01 & 3.19E-01 \\ \hline +%\textbf{D} & -1.13E-04 & -2.96E-02 & 7.15E-04 & 1.49E-02 & -1.21E-10 & 3.01E-09 & 1.12E+00 & 8.01E-01 \\ \hline +%\textbf{decay-A} & -1.28E-09 & -7.80E-11 & -1.44E-09 & -1.21E-10 & 7.00E-16 & -1.73E-17 & -1.26E-07 & 2.07E-07 \\ \hline +%\textbf{decay-B} & 5.14E-11 & -6.02E-09 & -4.11E-12 & 3.01E-09 & -1.73E-17 & 6.83E-16 & -1.86E-07 & 3.91E-08 \\ \hline +%\textbf{sigma-A} & -1.49E+01 & 7.00E+00 & 2.63E+01 & 1.12E+00 & -1.26E-07 & -1.86E-07 & 8.52E+04 & 1.79E+03 \\ \hline +%\textbf{sigma-B} & -3.74E-01 & -1.47E+00 & 3.19E-01 & 8.01E-01 & 2.07E-07 & 3.91E-08 & 1.79E+03 & 8.64E+04 \\ \hline +%\end{tabular} +%\end{table} +%\begin{table}[h!] +%\centering +%\caption{Correlation matrix.} +%\label{pearsonComputed} +%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} +%\hline +%{\ul \textit{\textbf{Correlation}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline +%\textbf{A} & 1.00E+00 & -5.02E-02 & -3.13E-01 & -1.03E-02 & -5.35E-01 & 2.17E-02 & -5.63E-01 & -1.40E-02 \\ \hline +%\textbf{B} & -5.02E-02 & 1.00E+00 & 8.47E-02 & -9.90E-01 & -1.20E-02 & -9.41E-01 & 9.80E-02 & -2.04E-02 \\ \hline +%\textbf{C} & -3.13E-01 & 8.47E-02 & 1.00E+00 & 5.37E-02 & -4.98E-01 & -1.44E-03 & 8.25E-01 & 9.96E-03 \\ \hline +%\textbf{D} & -1.03E-02 & -9.90E-01 & 5.37E-02 & 1.00E+00 & -3.75E-02 & 9.43E-01 & 3.14E-02 & 2.23E-02 \\ \hline +%\textbf{decay-A} & -5.35E-01 & -1.20E-02 & -4.98E-01 & -3.75E-02 & 1.00E+00 & -2.50E-02 & -1.64E-02 & 2.67E-02 \\ \hline +%\textbf{decay-B} & 2.17E-02 & -9.41E-01 & -1.44E-03 & 9.43E-01 & -2.50E-02 & 1.00E+00 & -2.44E-02 & 5.08E-03 \\ \hline +%\textbf{sigma-A} & -5.63E-01 & 9.80E-02 & 8.25E-01 & 3.14E-02 & -1.64E-02 & -2.44E-02 & 1.00E+00 & 2.08E-02 \\ \hline +%\textbf{sigma-B} & -1.40E-02 & -2.04E-02 & 9.96E-03 & 2.23E-02 & 2.67E-02 & 5.08E-03 & 2.08E-02 & 1.00E+00 \\ \hline +%\end{tabular} +%\end{table} +%\begin{table}[h!] +%\centering +%\caption{Variance Dependent Sensitivity matrix.} +%\label{VarDepSensitivityComputed} +%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} +%\hline +%{\ul \textit{\textbf{Variance Sensitivity}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline +%\textbf{A} & 1.00E+00 & -1.36E-01 & -3.77E-01 & -1.38E-02 & -1.56E-07 & 6.27E-09 & -1.82E+03 & -4.56E+01 \\ \hline +%\textbf{B} & -1.86E-02 & 1.00E+00 & 3.77E-02 & -4.94E-01 & -1.30E-09 & -1.00E-07 & 1.17E+02 & -2.45E+01 \\ \hline +%\textbf{C} & -2.60E-01 & 1.90E-01 & 1.00E+00 & 6.01E-02 & -1.21E-07 & -3.46E-10 & 2.21E+03 & 2.68E+01 \\ \hline +%\textbf{D} & -7.60E-03 & -1.99E+00 & 4.80E-02 & 1.00E+00 & -8.11E-09 & 2.02E-07 & 7.51E+01 & 5.37E+01 \\ \hline +%\textbf{decay-A} & -1.83E+06 & -1.11E+05 & -2.05E+06 & -1.73E+05 & 1.00E+00 & -2.47E-02 & -1.81E+08 & 2.96E+08 \\ \hline +%\textbf{decay-B} & 7.52E+04 & -8.82E+06 & -6.02E+03 & 4.40E+06 & -2.53E-02 & 1.00E+00 & -2.72E+08 & 5.72E+07 \\ \hline +%\textbf{sigma-A} & -1.75E-04 & 8.22E-05 & 3.08E-04 & 1.32E-05 & -1.48E-12 & -2.19E-12 & 1.00E+00 & 2.10E-02 \\ \hline +%\textbf{sigma-B} & -4.33E-06 & -1.70E-05 & 3.69E-06 & 9.27E-06 & 2.40E-12 & 4.52E-13 & 2.07E-02 & 1.00E+00 \\ \hline +%\end{tabular} +%\end{table} +%\begin{table}[h!] +%\centering +%\caption{Sensitivity matrix.} +%\label{SensitivityComputed} +%\begin{tabular}{|c|c|c|c|c|} +%\hline +%{\ul \textit{\textbf{Sensitivity (I/O)}}} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline +%\textbf{A} & 3.83E-06 & -1.78E-04 & -2.07E+04 & -1.86E+06 \\ \hline +%\textbf{B} & -1.36E-05 & 6.28E-05 & -8.80E+06 & -3.14E+05 \\ \hline +%\textbf{C} & 2.17E-06 & 3.05E-04 & 2.64E+04 & -2.00E+06 \\ \hline +%\textbf{D} & 6.96E-06 & 2.25E-05 & 4.40E+06 & -6.19E+04 \\ \hline +%\end{tabular} +%\end{table} +%\end{landscape} + From a2a1848dc353d12fd7d71e59851595f3c8a32374 Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Mon, 11 Jun 2018 11:47:02 -0600 Subject: [PATCH 3/9] smarter raven locator for qsubbing --- framework/CustomModes/MPISimulationMode.py | 8 +++++++- framework/raven_qsub_command.sh | 7 ++++--- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/framework/CustomModes/MPISimulationMode.py b/framework/CustomModes/MPISimulationMode.py index f7121e951c..e6b9fa7291 100644 --- a/framework/CustomModes/MPISimulationMode.py +++ b/framework/CustomModes/MPISimulationMode.py @@ -21,6 +21,7 @@ #End compatibility block for Python 3---------------------------------------------------------------- import os +import sys import math import string import Simulation @@ -66,6 +67,7 @@ def modifyInfo(self, runInfoDict): nodefile = os.environ["PBS_NODEFILE"] else: nodefile = self.__nodefile + self.raiseADebug('Setting up remote nodes based on "{}"'.format(nodefile)) lines = open(nodefile,"r").readlines() #XXX This is an undocumented way to pass information back newRunInfo['Nodes'] = list(lines) @@ -79,6 +81,7 @@ def modifyInfo(self, runInfoDict): newRunInfo['batchSize'] = maxBatchsize self.raiseAWarning("changing batchsize from "+str(oldBatchsize)+" to "+str(maxBatchsize)+" to fit on "+str(len(lines))+" processors") newBatchsize = newRunInfo['batchSize'] + self.raiseADebug('Batch size is "{}"'.format(newBatchsize)) if newBatchsize > 1: #need to split node lines so that numMPI nodes are available per run workingDir = runInfoDict['WorkingDir'] @@ -143,13 +146,16 @@ def __createAndRunQSUB(self, runInfoDict): jobName = jobName[:10]+'-'+jobName[-4:] print('JobName is limited to 15 characters; truncating to '+jobName) #Generate the qsub command needed to run input + ## find the raven_framework location + driverLocation = os.path.abspath(sys.argv[0]) + raven = os.path.abspath(os.path.join(os.path.dirname(driverLocation),'..','raven_framework')) command = ["qsub","-N",jobName]+\ runInfoDict["clusterParameters"]+\ ["-l", "select="+str(coresNeeded)+":ncpus="+str(ncpus)+":mpiprocs=1"+memString, "-l","walltime="+runInfoDict["expectedTime"], "-l","place="+self.__place,"-v", - 'COMMAND="../raven_framework '+ + 'COMMAND="{} '.format(raven)+ " ".join(runInfoDict["SimulationFiles"])+'"', runInfoDict['RemoteRunCommand']] #Change to frameworkDir so we find raven_qsub_command.sh diff --git a/framework/raven_qsub_command.sh b/framework/raven_qsub_command.sh index 3d86de1b4f..2db6410aa6 100755 --- a/framework/raven_qsub_command.sh +++ b/framework/raven_qsub_command.sh @@ -1,8 +1,9 @@ #!/bin/bash -if test -n "$PBS_O_WORKDIR"; then - cd $PBS_O_WORKDIR -fi +## FIXME why change to raven_framework? Stay in the working dir. +#if test -n "$PBS_O_WORKDIR"; then +# cd $PBS_O_WORKDIR +#fi which python which mpiexec From 5534036f8f3cc5d97cd785ab9662c33739a474aa Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Tue, 12 Jun 2018 08:04:13 -0600 Subject: [PATCH 4/9] implementation done --- framework/CustomModes/MPISimulationMode.py | 11 ++-- framework/Simulation.py | 1 + framework/raven_qsub_command.sh | 14 +++-- scripts/setup_raven_libs | 68 +++++++++++++++++++--- 4 files changed, 76 insertions(+), 18 deletions(-) diff --git a/framework/CustomModes/MPISimulationMode.py b/framework/CustomModes/MPISimulationMode.py index e6b9fa7291..0c8400ccfa 100644 --- a/framework/CustomModes/MPISimulationMode.py +++ b/framework/CustomModes/MPISimulationMode.py @@ -134,6 +134,7 @@ def __createAndRunQSUB(self, runInfoDict): else: memString = "" #batchSize = runInfoDict['batchSize'] + cwd = os.getcwd() frameworkDir = runInfoDict["FrameworkDir"] ncpus = runInfoDict['NumThreads'] jobName = runInfoDict['JobName'] if 'JobName' in runInfoDict.keys() else 'raven_qsub' @@ -147,20 +148,20 @@ def __createAndRunQSUB(self, runInfoDict): print('JobName is limited to 15 characters; truncating to '+jobName) #Generate the qsub command needed to run input ## find the raven_framework location - driverLocation = os.path.abspath(sys.argv[0]) - raven = os.path.abspath(os.path.join(os.path.dirname(driverLocation),'..','raven_framework')) + frameworkLocation = os.path.dirname(os.path.abspath(sys.argv[0])) + raven = os.path.abspath(os.path.join(frameworkLocation,'..','raven_framework')) command = ["qsub","-N",jobName]+\ runInfoDict["clusterParameters"]+\ ["-l", - "select="+str(coresNeeded)+":ncpus="+str(ncpus)+":mpiprocs=1"+memString, + "select={}:ncpus={}:mpiprocs=1{}".format(coresNeeded,ncpus,memString),#"+str(coresNeeded)+":ncpus="+str(ncpus)+":mpiprocs=1"+memString, "-l","walltime="+runInfoDict["expectedTime"], "-l","place="+self.__place,"-v", 'COMMAND="{} '.format(raven)+ " ".join(runInfoDict["SimulationFiles"])+'"', runInfoDict['RemoteRunCommand']] - #Change to frameworkDir so we find raven_qsub_command.sh + # Change to the working dir, so log files go to the right place remoteRunCommand = {} - remoteRunCommand["cwd"] = frameworkDir + remoteRunCommand["cwd"] = cwd #frameworkDir remoteRunCommand["args"] = command print("remoteRunCommand",remoteRunCommand) return remoteRunCommand diff --git a/framework/Simulation.py b/framework/Simulation.py index ddb7c322b1..b0ddc938c7 100644 --- a/framework/Simulation.py +++ b/framework/Simulation.py @@ -516,6 +516,7 @@ def initialize(self): @ Out, None """ #move the full simulation environment in the working directory + self.raiseADebug('Moving to working directory:',self.runInfoDict['WorkingDir']) os.chdir(self.runInfoDict['WorkingDir']) #add also the new working dir to the path sys.path.append(os.getcwd()) diff --git a/framework/raven_qsub_command.sh b/framework/raven_qsub_command.sh index 2db6410aa6..29fd3edc58 100755 --- a/framework/raven_qsub_command.sh +++ b/framework/raven_qsub_command.sh @@ -1,11 +1,17 @@ #!/bin/bash -## FIXME why change to raven_framework? Stay in the working dir. -#if test -n "$PBS_O_WORKDIR"; then -# cd $PBS_O_WORKDIR -#fi +if test -n "$PBS_O_WORKDIR"; then + echo Moving to working dir: ${PBS_O_WORKDIR} + cd $PBS_O_WORKDIR +fi +source activate raven_libraries + +echo 'python:' which python +echo '' +echo 'mpiexec:' which mpiexec +echo '' echo $COMMAND $COMMAND diff --git a/scripts/setup_raven_libs b/scripts/setup_raven_libs index 96edc818e4..5cb8e1904c 100755 --- a/scripts/setup_raven_libs +++ b/scripts/setup_raven_libs @@ -4,26 +4,76 @@ try_use_raven_environment () { #First check home directory for a script, then /opt/raven_libs - if test -e $HOME/.raven/environments/raven_libs_profile; + if test -e $HOME/.raven/environments/raven_libs_profile; + then + echo ... sourcing $HOME/.raven/environments/raven_libs_profile. + source $HOME/.raven/environments/raven_libs_profile + else + if test -e /opt/raven_libs/environments/raven_libs_profile; then - source $HOME/.raven/environments/raven_libs_profile + echo ... sourcing /opt/raven_libs/environments/raven_libs_profile + source /opt/raven_libs/environments/raven_libs_profile else - if test -e /opt/raven_libs/environments/raven_libs_profile; - then - source /opt/raven_libs/environments/raven_libs_profile - fi + echo ... no raven library environment found! Assuming already established. fi + fi } +echo Setting up raven python libraries ... + +# FIXME a lot of this code duplicates what's in establish_conda_env.sh, which extends +# what is done here in order to install libraries in the first place, and set up the +# conda environment. We should try to find a way to extract the common parts to +# reduce code duplication. + +# determine OS (for setting default conda location) +case $OSTYPE in + "linux-gnu") + OSOPTION="--linux" + ;; + "darwin"*) + OSOPTION="--mac" + ;; + "msys"*) + OSOPTION="--windows" + ;; + "cygwin"*) + OSOPTION="--windows" + ;; + *) + OSOPTION="" + ;; +esac + +# default location of conda definitions, windows is unsurprisingly an exception +## This is compatible with conda 4.4+, but not 4.3. +if [[ "$OSOPTION" = "--windows" ]]; +then + CONDA_DEFS="/c/ProgramData/Miniconda2/etc/profile.d/conda.sh"; +else + CONDA_DEFS="$HOME/miniconda2/etc/profile.d/conda.sh"; +fi + +# source conda definitions (for conda 4.4+) +echo Attempting to locate \"conda\" ... +if test -e ${CONDA_DEFS}; +then + echo Sourcing conda definitions at ${CONDA_DEFS} + source ${CONDA_DEFS} +fi + #Check for conda and activate raven_libraries -if which conda 2> /dev/null; +if command -v conda 2> /dev/null; then if conda env list | grep raven_libraries > /dev/null; then - source activate raven_libraries + echo ... activating conda environment raven_libraries. + conda activate raven_libraries else - try_use_raven_environment + echo ... conda known, but raven_libraries not found. Looking for raven environments ... + try_use_raven_environment fi else + echo ... conda unknown, looking for raven environments ... try_use_raven_environment fi From 0c89ccc5cc73863cef4a7c512e0b17b78d4412ef Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Tue, 12 Jun 2018 10:10:07 -0600 Subject: [PATCH 5/9] new test added --- .../CodeInterfaces/RAVEN/RAVENInterface.py | 4 +- framework/CustomModes/MPISimulationMode.py | 27 +-- framework/Simulation.py | 6 + framework/raven_qsub_command.sh | 9 +- .../ext_dataobjects.xml | 34 ++++ .../testConversionModule.py | 31 ++++ .../test_rom_trainer.xml | 174 ++++++++++++++++++ .../test_rom_trainer/TMI_fake.py | 27 +++ .../test_raven_running_raven_int_models.xml | 131 +++++++++++++ tests/cluster_tests/test_pbspro.sh | 48 +---- tests/cluster_tests/test_qsubs.sh | 79 +++----- 11 files changed, 455 insertions(+), 115 deletions(-) create mode 100644 tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/ext_dataobjects.xml create mode 100755 tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/testConversionModule.py create mode 100644 tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer.xml create mode 100644 tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer/TMI_fake.py create mode 100644 tests/cluster_tests/RavenRunsRaven/test_raven_running_raven_int_models.xml diff --git a/framework/CodeInterfaces/RAVEN/RAVENInterface.py b/framework/CodeInterfaces/RAVEN/RAVENInterface.py index 1d2d276094..3f02fca7fd 100644 --- a/framework/CodeInterfaces/RAVEN/RAVENInterface.py +++ b/framework/CodeInterfaces/RAVEN/RAVENInterface.py @@ -95,12 +95,12 @@ def _readMoreXML(self,xmlNode): # check for existence if not os.path.exists(source): raise IOError(self.printTag+' ERROR: the conversionModule "{}" was not found!' - .format(self.extModForVarsManipulationPath)) + .format(source)) # check module is imported checkImport = utils.importFromPath(source) if checkImport is None: raise IOError(self.printTag+' ERROR: the conversionModule "{}" failed on import!' - .format(self.extModForVarsManipulationPath)) + .format(source)) # check methods are in place noScalar = 'convertNotScalarSampledVariables' in checkImport.__dict__ scalar = 'manipulateScalarSampledVariables' in checkImport.__dict__ diff --git a/framework/CustomModes/MPISimulationMode.py b/framework/CustomModes/MPISimulationMode.py index 0c8400ccfa..14e98637c5 100644 --- a/framework/CustomModes/MPISimulationMode.py +++ b/framework/CustomModes/MPISimulationMode.py @@ -124,21 +124,23 @@ def __createAndRunQSUB(self, runInfoDict): @ Out, remoteRunCommand, dict, dictionary of command. """ # Check if the simulation has been run in PBS mode and, in case, construct the proper command - #while true, this is not the number that we want to select + # determine the cores needed for the job if self.__coresNeeded is not None: coresNeeded = self.__coresNeeded else: coresNeeded = runInfoDict['batchSize']*runInfoDict['NumMPI'] + # get the requested memory, if any if self.__memNeeded is not None: memString = ":mem="+self.__memNeeded else: memString = "" - #batchSize = runInfoDict['batchSize'] - cwd = os.getcwd() + # raven/framework location frameworkDir = runInfoDict["FrameworkDir"] + # number of "threads" ncpus = runInfoDict['NumThreads'] + # job title jobName = runInfoDict['JobName'] if 'JobName' in runInfoDict.keys() else 'raven_qsub' - #check invalid characters + ## fix up job title validChars = set(string.ascii_letters).union(set(string.digits)).union(set('-_')) if any(char not in validChars for char in jobName): raise IOError('JobName can only contain alphanumeric and "_", "-" characters! Received'+jobName) @@ -146,23 +148,26 @@ def __createAndRunQSUB(self, runInfoDict): if len(jobName) > 15: jobName = jobName[:10]+'-'+jobName[-4:] print('JobName is limited to 15 characters; truncating to '+jobName) - #Generate the qsub command needed to run input - ## find the raven_framework location - frameworkLocation = os.path.dirname(os.path.abspath(sys.argv[0])) - raven = os.path.abspath(os.path.join(frameworkLocation,'..','raven_framework')) + # Generate the qsub command needed to run input + ## raven_framework location + raven = os.path.abspath(os.path.join(frameworkDir,'..','raven_framework')) + ## generate the command, which will be passed into "args" of subprocess.call command = ["qsub","-N",jobName]+\ runInfoDict["clusterParameters"]+\ ["-l", - "select={}:ncpus={}:mpiprocs=1{}".format(coresNeeded,ncpus,memString),#"+str(coresNeeded)+":ncpus="+str(ncpus)+":mpiprocs=1"+memString, + "select={}:ncpus={}:mpiprocs=1{}".format(coresNeeded,ncpus,memString), "-l","walltime="+runInfoDict["expectedTime"], "-l","place="+self.__place,"-v", 'COMMAND="{} '.format(raven)+ " ".join(runInfoDict["SimulationFiles"])+'"', runInfoDict['RemoteRunCommand']] - # Change to the working dir, so log files go to the right place + # Set parameters for the run command remoteRunCommand = {} - remoteRunCommand["cwd"] = cwd #frameworkDir + ## directory to start in, where the input file is + remoteRunCommand["cwd"] = runInfoDict['InputDir'] + ## command to run in that directory remoteRunCommand["args"] = command + ## print out for debugging print("remoteRunCommand",remoteRunCommand) return remoteRunCommand diff --git a/framework/Simulation.py b/framework/Simulation.py index b0ddc938c7..41bba4ac99 100644 --- a/framework/Simulation.py +++ b/framework/Simulation.py @@ -609,6 +609,9 @@ def __readRunInfo(self,xmlNode,runInfoSkip,xmlFilename): else: self.runInfoDict['printInput'] = text+'.xml' elif element.tag == 'WorkingDir': + # first store the cwd, the "CallDir" + self.runInfoDict['CallDir'] = os.getcwd() + # then get the requested "WorkingDir" tempName = element.text if '~' in tempName: tempName = os.path.expanduser(tempName) @@ -619,8 +622,11 @@ def __readRunInfo(self,xmlNode,runInfoSkip,xmlFilename): else: if xmlFilename == None: self.raiseAnError(IOError,'Relative working directory requested but xmlFilename is None.') + # store location of the input xmlDirectory = os.path.dirname(os.path.abspath(xmlFilename)) + self.runInfoDict['InputDir'] = xmlDirectory rawRelativeWorkingDir = element.text.strip() + # working dir is file location + relative working dir self.runInfoDict['WorkingDir'] = os.path.join(xmlDirectory,rawRelativeWorkingDir) utils.makeDir(self.runInfoDict['WorkingDir']) elif element.tag == 'maxQueueSize': diff --git a/framework/raven_qsub_command.sh b/framework/raven_qsub_command.sh index 29fd3edc58..8fb656f67c 100755 --- a/framework/raven_qsub_command.sh +++ b/framework/raven_qsub_command.sh @@ -5,11 +5,10 @@ if test -n "$PBS_O_WORKDIR"; then cd $PBS_O_WORKDIR fi -source activate raven_libraries - -echo 'python:' -which python -echo '' +# "which python" does not necessarily show "conda" if conda hasn't been activated yet +#echo 'python:' +#which python +#echo '' echo 'mpiexec:' which mpiexec echo '' diff --git a/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/ext_dataobjects.xml b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/ext_dataobjects.xml new file mode 100644 index 0000000000..547f6d322b --- /dev/null +++ b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/ext_dataobjects.xml @@ -0,0 +1,34 @@ + + + DeltaTimeScramToAux,DG1recoveryTime + OutputPlaceHolder + + + DeltaTimeScramToAux,DG1recoveryTime + CladTempThreshold,time + + + DeltaTimeScramToAux,DG1recoveryTime + CladTempThreshold + + + DeltaTimeScramToAux,DG1recoveryTime + OutputPlaceHolder + + + DeltaTimeScramToAux,DG1recoveryTime + OutputPlaceHolder + + + DeltaTimeScramToAux,DG1recoveryTime + + + + DeltaTimeScramToAux,DG1recoveryTime + CladTempThreshold,time + + + DeltaTimeScramToAux,DG1recoveryTime + CladTempThreshold + + diff --git a/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/testConversionModule.py b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/testConversionModule.py new file mode 100755 index 0000000000..5cf836b60f --- /dev/null +++ b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/testConversionModule.py @@ -0,0 +1,31 @@ +# Copyright 2017 Battelle Energy Alliance, LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + + +def manipulateScalarSampledVariables(sampledVars): + """ + This method is aimed to manipulate scalar variables. + The user can create new variables based on the + variables sampled by RAVEN + @ In, sampledVars, dict, dictionary of + sampled variables ({"var1":value1,"var2":value2}) + @ Out, None, the new variables should be + added in the "sampledVariables" dictionary + """ + sampledVars['Models|ROM@subType:SciKitLearn@name:ROM1|coef0'] = sampledVars['Models|ROM@subType:SciKitLearn@name:ROM1|C']/10.0 + + + + diff --git a/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer.xml b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer.xml new file mode 100644 index 0000000000..65c38b0adf --- /dev/null +++ b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer.xml @@ -0,0 +1,174 @@ + + + + + test_rom_trainer + MC_for_rom_trainer, test_extract_for_rom_trainer, + test_rom_trainer, test_rom_trainerHS, + rom_MC, rom_MCHS, + test_rom_trainer_nd_interp,rom_MC_nd_interpolator + 1 + + + + + 0 + 2000 + + + 0 + 1000 + + + + + + DeltaTimeScramToAux,DG1recoveryTime,time,CladTempThreshold + + + DeltaTimeScramToAux,DG1recoveryTime + CladTempThreshold + svm|SVR + linear + 10.0 + 0.0001 + 0.0 + + + DeltaTimeScramToAux,DG1recoveryTime + CladTempThreshold,time + svm|SVR + linear + 10.0 + 0.0001 + 0.0 + + + DeltaTimeScramToAux,DG1recoveryTime + CladTempThreshold +

3

+ + + + + + + 3 + + + auxbackup + + 500 + + + + auxbackup + 0.5 1.0 0.0 + + 500 + + + + 3 + + + auxbackup + + + DG1backup + + + + + 3 + 200286 + + + auxbackup + + + DG1backup + + + + + + + inputHolder + PythonModule + RAVENmcCode3 + MC_TEST_EXTRACT_STEP_FOR_ROM_TRAINER + + + MC_TEST_EXTRACT_STEP_FOR_ROM_TRAINER + MC_TEST_EXTRACT_STEP_FOR_ROM_TRAINER + Pointset_from_database_for_rom_trainer + Historyset_from_database_for_rom_trainer + ciccio + ciccioHS + + + Pointset_from_database_for_rom_trainer + ROM1 + + + Historyset_from_database_for_rom_trainer + ROMHS + + + Pointset_from_database_for_rom_trainer + ROM2 + + + data_for_sampling_empty_at_begin + ROM1 + gridRom + outputMontecarloRom + outputMontecarloRom_dump + + + data_for_sampling_empty_at_begin + ROMHS + gridRom + outputMontecarloRomHS + outputMontecarloRomHS_dump + + + data_for_sampling_empty_at_begin_nd + ROM2 + RAVENmcND + outputMontecarloRomND + outputMontecarloRomND_dump + + + + + + csv + outputMontecarloRom + + + csv + outputMontecarloRomHS + + + csv + outputMontecarloRomND + + + csv + Pointset_from_database_for_rom_trainer + + + csv + Historyset_from_database_for_rom_trainer + + + + + + + + + + diff --git a/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer/TMI_fake.py b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer/TMI_fake.py new file mode 100644 index 0000000000..5243f6b8e1 --- /dev/null +++ b/tests/cluster_tests/RavenRunsRaven/raven_running_raven_internal_models/test_rom_trainer/TMI_fake.py @@ -0,0 +1,27 @@ +# Copyright 2017 Battelle Energy Alliance, LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import math +import numpy + +def run(self, Input): + number_of_steps = 16 + self.time = numpy.zeros(number_of_steps) + DeltaTimeScramToAux = Input["DeltaTimeScramToAux"] + DG1recoveryTime = Input["DG1recoveryTime"] + self.CladTempThreshold = numpy.zeros(number_of_steps) + self.UpperPlenumEnergy= numpy.zeros(number_of_steps) + for i in range(len(self.time)): + self.time[i] = 0.25*i + self.CladTempThreshold[i] = self.time[i]*50.0 + DeltaTimeScramToAux*200.0 + DG1recoveryTime*500.0 + self.UpperPlenumEnergy[i] = self.time[i]*5.0 + DeltaTimeScramToAux*30.0 + DG1recoveryTime*40.0 + DeltaTimeScramToAux*DG1recoveryTime*5.0 diff --git a/tests/cluster_tests/RavenRunsRaven/test_raven_running_raven_int_models.xml b/tests/cluster_tests/RavenRunsRaven/test_raven_running_raven_int_models.xml new file mode 100644 index 0000000000..6779cd9e22 --- /dev/null +++ b/tests/cluster_tests/RavenRunsRaven/test_raven_running_raven_int_models.xml @@ -0,0 +1,131 @@ + + + + framework/CodeInterfaceTests.RAVENrunningRAVEN_ROM + alfoa + 2017-09-16 + Models.Code.RAVEN + + This test is aimed to check the functionality of the RAVEN code interface (RAVEN running RAVEN). It tests the MPI implementation of the + SLAVE RAVEN runs. In this case, 3 simultaneous SLAVE RAVEN (batchSize=3) runs are going to be spawned, each of them using 2 processors (NumMPI=2). + The NumMPI XML node MUST BE INPUTTED if the SLAVE RAVEN runs must run in multiple processors! + + + + + raven_running_raven_internal_models + FirstMRun + + 5 + mpi + 0:10:0 + -P nst -j oe + + + + test_rom_trainer.xml + ext_dataobjects.xml + + + + + + test_rom_trainer.xml + ext_dataobjects.xml + raven_running_rom + MC_external + testPrintHistorySet + test_external_db + testPrintHistorySet_dump + testPointSet + testPointSet_dump + + + + + + + %FRAMEWORK_DIR%/../raven_framework + outputMontecarloRom_dump,outputMontecarloRomHS_dump + + + Models|ROM@subType:SciKitLearn@name:ROM1|C + + + Models|ROM@subType:SciKitLearn@name:ROM1|C + Models|ROM@subType:SciKitLearn@name:ROM1|tol + Samplers|Grid@name:gridRom|constant@name:DG1recoveryTime + + + + + + + + 4 + 0.1 + + + 0.0001 + 0.000001 + + + 0 + 1000 + + + + + + + + 5 + + + C_distrib + + + toll_distrib + + + DG1backup + + + + + + + + + + + + + csv + testPrintHistorySet + input,output + + + csv + testPointSet + input,output + + + + + + + loss_factor,tollerance + OutputPlaceHolder + + + loss_factor,DeltaTimeScramToAux,tollerance,DG1recoveryTime + CladTempThreshold + + + loss_factor,DeltaTimeScramToAux,tollerance,DG1recoveryTime + CladTempThreshold + + + + diff --git a/tests/cluster_tests/test_pbspro.sh b/tests/cluster_tests/test_pbspro.sh index 3160bea515..0757e8e2e7 100755 --- a/tests/cluster_tests/test_pbspro.sh +++ b/tests/cluster_tests/test_pbspro.sh @@ -46,46 +46,32 @@ wait_lines () } rm -Rf FirstMQRun/ - #REQUIREMENT_TEST R-IS-7 python ../../framework/Driver.py test_mpiqsub_local.xml pbspro_mpi.xml cluster_runinfo.xml - wait_lines 'FirstMQRun/[1-6]/*test.csv' 6 mpiqsub rm -Rf FirstMNRun/ - python ../../framework/Driver.py test_mpiqsub_nosplit.xml cluster_runinfo.xml - wait_lines 'FirstMNRun/[1-6]/*.csv' 6 mpiqsub_nosplit rm -Rf FirstMLRun/ - python ../../framework/Driver.py test_mpiqsub_limitnode.xml cluster_runinfo.xml - wait_lines 'FirstMLRun/[1-6]/*.csv' 6 mpiqsub_limitnode rm -Rf FirstMRun/ - qsub -P moose -l select=6:ncpus=4:mpiprocs=1 -l walltime=10:00:00 -l place=free -W block=true ./run_mpi_test.sh - wait_lines 'FirstMRun/[1-6]/*test.csv' 6 mpi rm -Rf FirstPRun/ - python ../../framework/Driver.py test_pbs.xml cluster_runinfo.xml - wait_lines 'FirstPRun/[1-6]/*test.csv' 6 pbsdsh rm -Rf FirstMFRun/ - python ../../framework/Driver.py test_mpiqsub_flex.xml cluster_runinfo.xml - wait_lines 'FirstMFRun/[1-6]/*.csv' 6 mpiqsub_flex rm -Rf FirstMForcedRun/ - python ../../framework/Driver.py test_mpiqsub_forced.xml cluster_runinfo.xml - wait_lines 'FirstMForcedRun/[1-6]/*.csv' 6 mpiqsub_forced ###################################### @@ -94,84 +80,60 @@ wait_lines 'FirstMForcedRun/[1-6]/*.csv' 6 mpiqsub_forced # first stes (external model in parallel) cd InternalParallel/ rm -Rf InternalParallelExtModel/*.csv - #REQUIREMENT_TEST R-IS-8 python ../../../framework/Driver.py test_internal_parallel_extModel.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'InternalParallelExtModel/*.csv' 28 paralExtModel - cd .. # second test (ROM in parallel) cd InternalParallel/ rm -Rf InternalParallelScikit/*.csv - #REQUIREMENT_TEST R-IS-9 python ../../../framework/Driver.py test_internal_parallel_ROM_scikit.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'InternalParallelScikit/*.csv' 2 paralROM - cd .. # third test (PostProcessor in parallel) cd InternalParallel/ rm -Rf InternalParallelPostProcessorLS/*.csv - python ../../../framework/Driver.py test_internal_parallel_PP_LS.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'InternalParallelPostProcessorLS/*.csv' 4 parallelPP - cd .. # forth test (Topology Picard in parallel) cd InternalParallel/ rm -Rf InternalParallelMSR/*.csv - python ../../../framework/Driver.py test_internal_MSR.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'InternalParallelMSR/*.csv' 1 parallelMSR - cd .. # fifth test (Ensamble Model Picard in parallel) cd InternalParallel/ rm -Rf metaModelNonLinearParallel/*.png - python ../../../framework/Driver.py test_ensemble_model_picard_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'metaModelNonLinearParallel/*.png' 3 parallelEnsemblePicard - cd .. # sixth test (Ensamble Model Linear Picard in parallel) cd InternalParallel/ rm -Rf metaModelLinearParallel/*.png - python ../../../framework/Driver.py test_ensemble_model_linear_internal_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'metaModelLinearParallel/*.png' 2 parallelEnsembleLinear - cd .. # seven test (HybridModel Code in parallel) cd InternalParallel/ rm -Rf hybridModelCode/*.csv - python ../../../framework/Driver.py test_hybrid_model_code.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'hybridModelCode/*.csv' 1 parallelHybridModelCode - cd .. # eighth test (HybridModel External Model in parallel) cd InternalParallel/ rm -Rf hybridModelExternal/*.csv - python ../../../framework/Driver.py test_hybrid_model_external.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'hybridModelExternal/*.csv' 1 parallelHybridModelExternal - cd .. ############################################ @@ -184,13 +146,17 @@ cd .. cd AdaptiveSobol/ rm -Rf workdir/* - python ../../../framework/Driver.py test_adapt_sobol_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo.xml - wait_lines 'workdir/*.csv' 1 adaptiveSobol - cd .. +# Raven-Running-Raven (RAVEN code interface) +cd RavenRunsRaven/raven_running_raven_internal_models/ +rm -Rf FirstMRun DatabaseStorage *csv testPointSet_dump.xml +cd .. +../../../raven_framework test_raven_running_raven_int_models.xml ../pbspro_mpi.xml ../cluster_runinfo.xml +wait_lines 'raven_running_raven_internal_models/testP*.csv' 17 ravenRunningRaven +cd .. if test $num_fails -eq 0; then echo ALL PASSED diff --git a/tests/cluster_tests/test_qsubs.sh b/tests/cluster_tests/test_qsubs.sh index 070a834d9c..db02f5085b 100755 --- a/tests/cluster_tests/test_qsubs.sh +++ b/tests/cluster_tests/test_qsubs.sh @@ -44,46 +44,32 @@ wait_lines () } rm -Rf FirstMQRun/ - #REQUIREMENT_TEST R-IS-7 -python ../../framework/Driver.py test_mpiqsub_local.xml pbspro_mpi.xml cluster_runinfo_legacy.xml - +../../raven_framework test_mpiqsub_local.xml pbspro_mpi.xml cluster_runinfo_legacy.xml wait_lines 'FirstMQRun/[1-6]/*test.csv' 6 mpiqsub rm -Rf FirstMNRun/ - -python ../../framework/Driver.py test_mpiqsub_nosplit.xml cluster_runinfo_legacy.xml - +../../raven_framework test_mpiqsub_nosplit.xml cluster_runinfo_legacy.xml wait_lines 'FirstMNRun/[1-6]/*.csv' 6 mpiqsub_nosplit rm -Rf FirstMLRun/ - -python ../../framework/Driver.py test_mpiqsub_limitnode.xml cluster_runinfo_legacy.xml - +../../raven_framework test_mpiqsub_limitnode.xml cluster_runinfo_legacy.xml wait_lines 'FirstMLRun/[1-6]/*.csv' 6 mpiqsub_limitnode rm -Rf FirstMRun/ - qsub -P moose -l select=6:ncpus=4:mpiprocs=1 -l walltime=10:00:00 -l place=free -W block=true ./run_mpi_test.sh - wait_lines 'FirstMRun/[1-6]/*test.csv' 6 mpi rm -Rf FirstPRun/ - -python ../../framework/Driver.py test_pbs.xml cluster_runinfo_legacy.xml - +../../raven_framework test_pbs.xml cluster_runinfo_legacy.xml wait_lines 'FirstPRun/[1-6]/*test.csv' 6 pbsdsh rm -Rf FirstMFRun/ - -python ../../framework/Driver.py test_mpiqsub_flex.xml cluster_runinfo_legacy.xml - +../../raven_framework test_mpiqsub_flex.xml cluster_runinfo_legacy.xml wait_lines 'FirstMFRun/[1-6]/*.csv' 6 mpiqsub_flex rm -Rf FirstMForcedRun/ - -python ../../framework/Driver.py test_mpiqsub_forced.xml cluster_runinfo_legacy.xml - +../../raven_framework test_mpiqsub_forced.xml cluster_runinfo_legacy.xml wait_lines 'FirstMForcedRun/[1-6]/*.csv' 6 mpiqsub_forced ###################################### @@ -92,84 +78,59 @@ wait_lines 'FirstMForcedRun/[1-6]/*.csv' 6 mpiqsub_forced # first stes (external model in parallel) cd InternalParallel/ rm -Rf InternalParallelExtModel/*.csv - #REQUIREMENT_TEST R-IS-8 -python ../../../framework/Driver.py test_internal_parallel_extModel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_internal_parallel_extModel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelExtModel/*.csv' 28 paralExtModel - cd .. # second test (ROM in parallel) cd InternalParallel/ rm -Rf InternalParallelScikit/*.csv - #REQUIREMENT_TEST R-IS-9 -python ../../../framework/Driver.py test_internal_parallel_ROM_scikit.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_internal_parallel_ROM_scikit.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelScikit/*.csv' 2 paralROM - cd .. # third test (PostProcessor in parallel) cd InternalParallel/ rm -Rf InternalParallelPostProcessorLS/*.csv - -python ../../../framework/Driver.py test_internal_parallel_PP_LS.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_internal_parallel_PP_LS.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelPostProcessorLS/*.csv' 4 parallelPP - cd .. # forth test (Topology Picard in parallel) - cd InternalParallel/ rm -Rf InternalParallelMSR/*.csv - -python ../../../framework/Driver.py test_internal_MSR.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_internal_MSR.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelMSR/*.csv' 1 parallelMSR - cd .. # fifth test (Ensamble Model Picard in parallel) cd InternalParallel/ rm -Rf metaModelNonLinearParallel/*.png - -python ../../../framework/Driver.py test_ensemble_model_picard_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_ensemble_model_picard_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'metaModelNonLinearParallel/*.png' 3 parallelEnsemblePicard - cd .. # sixth test (Ensamble Model Picard in parallel) cd InternalParallel/ rm -Rf metaModelLinearParallel/*.png - -python ../../../framework/Driver.py test_ensemble_model_linear_internal_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_ensemble_model_linear_internal_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'metaModelLinearParallel/*.png' 2 parallelEnsembleLinear - cd .. # seven test (HybridModel Code in parallel) cd InternalParallel/ rm -Rf hybridModelCode/*.csv - -python ../../../framework/Driver.py test_hybrid_model_code.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_hybrid_model_code.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'hybridModelCode/*.csv' 1 parallelHybridModelCode - cd .. # eighth test (HybridModel External Model in parallel) cd InternalParallel/ rm -Rf hybridModelExternal/*.csv - -python ../../../framework/Driver.py test_hybrid_model_external.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_hybrid_model_external.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'hybridModelExternal/*.csv' 1 parallelHybridModelExternal - cd .. ############################################ @@ -180,13 +141,19 @@ cd .. # other parallel objects tests # ################################ +# Adaptive Sobol cd AdaptiveSobol/ rm -Rf workdir/* - -python ../../../framework/Driver.py test_adapt_sobol_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml - +../../../raven_framework test_adapt_sobol_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'workdir/*.csv' 1 adaptiveSobol +cd .. +# Raven-Running-Raven (RAVEN code interface) +cd RavenRunsRaven/raven_running_raven_internal_models/ +rm -Rf FirstMRun DatabaseStorage *csv testPointSet_dump.xml +cd .. +../../../raven_framework test_raven_running_raven_int_models.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml +wait_lines 'raven_running_raven_internal_models/testP*.csv' 17 ravenRunningRaven cd .. From 8887b8ff92cc3c25653d291051a0379377fbfe85 Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Tue, 12 Jun 2018 10:32:51 -0600 Subject: [PATCH 6/9] test included --- tests/cluster_tests/test_pbspro.sh | 6 +++--- tests/cluster_tests/test_qsubs.sh | 23 ++++++++++++++++++++--- 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/tests/cluster_tests/test_pbspro.sh b/tests/cluster_tests/test_pbspro.sh index 0757e8e2e7..58aba28786 100755 --- a/tests/cluster_tests/test_pbspro.sh +++ b/tests/cluster_tests/test_pbspro.sh @@ -37,11 +37,11 @@ wait_lines () fails=$fails', '$NAME num_fails=$(($num_fails+1)) printf '\n\nStandard Error:\n' - cat $RAVEN_FRAMEWORK_DIR/test_qsub.e* + cat test_qsub.e* printf '\n\nStandard Output:\n' - cat $RAVEN_FRAMEWORK_DIR/test_qsub.o* + cat test_qsub.o* fi - rm $RAVEN_FRAMEWORK_DIR/test_qsub.[eo]* + rm test_qsub.[eo]* } diff --git a/tests/cluster_tests/test_qsubs.sh b/tests/cluster_tests/test_qsubs.sh index db02f5085b..64bb724ef2 100755 --- a/tests/cluster_tests/test_qsubs.sh +++ b/tests/cluster_tests/test_qsubs.sh @@ -35,11 +35,11 @@ wait_lines () fails=$fails', '$NAME num_fails=$(($num_fails+1)) printf '\n\nStandard Error:\n' - cat $RAVEN_FRAMEWORK_DIR/test_qsub.e* + cat test_qsub.e* printf '\n\nStandard Output:\n' - cat $RAVEN_FRAMEWORK_DIR/test_qsub.o* + cat test_qsub.o* fi - rm $RAVEN_FRAMEWORK_DIR/test_qsub.[eo]* + rm test_qsub.[eo]* } @@ -47,30 +47,37 @@ rm -Rf FirstMQRun/ #REQUIREMENT_TEST R-IS-7 ../../raven_framework test_mpiqsub_local.xml pbspro_mpi.xml cluster_runinfo_legacy.xml wait_lines 'FirstMQRun/[1-6]/*test.csv' 6 mpiqsub +echo '' rm -Rf FirstMNRun/ ../../raven_framework test_mpiqsub_nosplit.xml cluster_runinfo_legacy.xml wait_lines 'FirstMNRun/[1-6]/*.csv' 6 mpiqsub_nosplit +echo '' rm -Rf FirstMLRun/ ../../raven_framework test_mpiqsub_limitnode.xml cluster_runinfo_legacy.xml wait_lines 'FirstMLRun/[1-6]/*.csv' 6 mpiqsub_limitnode +echo '' rm -Rf FirstMRun/ qsub -P moose -l select=6:ncpus=4:mpiprocs=1 -l walltime=10:00:00 -l place=free -W block=true ./run_mpi_test.sh wait_lines 'FirstMRun/[1-6]/*test.csv' 6 mpi +echo '' rm -Rf FirstPRun/ ../../raven_framework test_pbs.xml cluster_runinfo_legacy.xml wait_lines 'FirstPRun/[1-6]/*test.csv' 6 pbsdsh +echo '' rm -Rf FirstMFRun/ ../../raven_framework test_mpiqsub_flex.xml cluster_runinfo_legacy.xml wait_lines 'FirstMFRun/[1-6]/*.csv' 6 mpiqsub_flex +echo '' rm -Rf FirstMForcedRun/ ../../raven_framework test_mpiqsub_forced.xml cluster_runinfo_legacy.xml wait_lines 'FirstMForcedRun/[1-6]/*.csv' 6 mpiqsub_forced +echo '' ###################################### # test parallel for internal Objects # @@ -82,6 +89,7 @@ rm -Rf InternalParallelExtModel/*.csv ../../../raven_framework test_internal_parallel_extModel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelExtModel/*.csv' 28 paralExtModel cd .. +echo '' # second test (ROM in parallel) cd InternalParallel/ @@ -90,6 +98,7 @@ rm -Rf InternalParallelScikit/*.csv ../../../raven_framework test_internal_parallel_ROM_scikit.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelScikit/*.csv' 2 paralROM cd .. +echo '' # third test (PostProcessor in parallel) cd InternalParallel/ @@ -97,6 +106,7 @@ rm -Rf InternalParallelPostProcessorLS/*.csv ../../../raven_framework test_internal_parallel_PP_LS.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelPostProcessorLS/*.csv' 4 parallelPP cd .. +echo '' # forth test (Topology Picard in parallel) cd InternalParallel/ @@ -104,6 +114,7 @@ rm -Rf InternalParallelMSR/*.csv ../../../raven_framework test_internal_MSR.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'InternalParallelMSR/*.csv' 1 parallelMSR cd .. +echo '' # fifth test (Ensamble Model Picard in parallel) cd InternalParallel/ @@ -111,6 +122,7 @@ rm -Rf metaModelNonLinearParallel/*.png ../../../raven_framework test_ensemble_model_picard_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'metaModelNonLinearParallel/*.png' 3 parallelEnsemblePicard cd .. +echo '' # sixth test (Ensamble Model Picard in parallel) cd InternalParallel/ @@ -118,6 +130,7 @@ rm -Rf metaModelLinearParallel/*.png ../../../raven_framework test_ensemble_model_linear_internal_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'metaModelLinearParallel/*.png' 2 parallelEnsembleLinear cd .. +echo '' # seven test (HybridModel Code in parallel) cd InternalParallel/ @@ -125,6 +138,7 @@ rm -Rf hybridModelCode/*.csv ../../../raven_framework test_hybrid_model_code.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'hybridModelCode/*.csv' 1 parallelHybridModelCode cd .. +echo '' # eighth test (HybridModel External Model in parallel) cd InternalParallel/ @@ -132,6 +146,7 @@ rm -Rf hybridModelExternal/*.csv ../../../raven_framework test_hybrid_model_external.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'hybridModelExternal/*.csv' 1 parallelHybridModelExternal cd .. +echo '' ############################################ # test parallel for internal Objects ENDED # @@ -147,6 +162,7 @@ rm -Rf workdir/* ../../../raven_framework test_adapt_sobol_parallel.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'workdir/*.csv' 1 adaptiveSobol cd .. +echo '' # Raven-Running-Raven (RAVEN code interface) cd RavenRunsRaven/raven_running_raven_internal_models/ @@ -155,6 +171,7 @@ cd .. ../../../raven_framework test_raven_running_raven_int_models.xml ../pbspro_mpi.xml ../cluster_runinfo_legacy.xml wait_lines 'raven_running_raven_internal_models/testP*.csv' 17 ravenRunningRaven cd .. +echo '' if test $num_fails -eq 0; then From 9504f01d17092cc543b58969ab6f64bf31e78c1d Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Tue, 12 Jun 2018 10:55:33 -0600 Subject: [PATCH 7/9] cleanup --- doc/user_guide/statisticsToDataObject.tex | 296 ------------------ .../StatisticalAnalysis/comparingSamplers.xml | 233 -------------- .../comparingSamplers/meanPlotter_scatter.pdf | Bin 11430 -> 0 bytes .../comparingSamplers/referenceInput.xml | 39 --- .../comparingSamplers/varPlotter_scatter.pdf | Bin 11892 -> 0 bytes 5 files changed, 568 deletions(-) delete mode 100644 doc/user_guide/statisticsToDataObject.tex delete mode 100644 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml delete mode 100644 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter.pdf delete mode 100755 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/referenceInput.xml delete mode 100644 tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter.pdf diff --git a/doc/user_guide/statisticsToDataObject.tex b/doc/user_guide/statisticsToDataObject.tex deleted file mode 100644 index 1d3321cad8..0000000000 --- a/doc/user_guide/statisticsToDataObject.tex +++ /dev/null @@ -1,296 +0,0 @@ -\subsection{Plotting Multiple Statistical Analyses} -In section \ref{sec:SAraven} we covered how to perform statistical analysis and print the results to an XML -file. However, what if we want to use the statistics calculated in more RAVEN calculations, like plotting and -further postprocessing? - -In the following two sections, we consider two use cases. -\begin{itemize} - \item In the first case, we want to compare the statistics obtained by three different sampler types on the - same model. We want to run the sampling, analyze the statistics, then make two plots where we can see how - the means and variances of the data compare between the sampling strategies. The plots will have the - sampler type on the x-axis, and the metric values on the y-axis. - \item In the second case, we want to see the time evolution of the statistics; that is, the mean and 5/95 - percentile of the model output as it evolves in time. We want to plot this so that the x-axis is time, - and the y-axis is the model output value, with three series, one each for fifth percentile, mean, and - ninety-fifth percentile. -\end{itemize} - -Note that while we use the `BasicStatistics` postprocessor as an example, there are many other RAVEN -postprocessors that produce XML outputs that follow this same strategy. - -\subsubsection{Case 1: Comparing Samplers} -In this case, we are interested in comparing the statistics obtained by several samplers (Monte Carlo, Grid, -and Stratified/Latin Hypercube) on a plot. To do -this, we need to complete the following steps: -\begin{enumerate} - \item Sample the model using Monte Carlo, Grid, and Stratified samplers, storing results in separate point - sets. - \item Perform statistical analysis (mean and variance) on each of the point sets containing the samples, writing results to - separate RAVEN XML output files. - \item Read in the RAVEN XML output files, creating a point set with the sampler types as inputs and the mean - and variance as outputs. - \item Plot the metrics as a function of the sampler used. -\end{enumerate} - -These steps lead to the following \xmlNode{RunInfo} block, see especially the \xmlNode{Sequence}: -\xmlExample{framework/user_guide/StatisticalAnalysis/comparingSamplers.xml}{RunInfo} -Note that each sampling and statistics requires its own step, which grows our total steps from four -conceptually to eight in RAVEN. - -Next, let's define the \xmlNode{Steps} block that will execute this \xmlNode{sequence}: -\xmlExample{framework/user_guide/StatisticalAnalysis/comparingSamplers.xml}{Steps} - -The three ``sample'' steps take the code input (\xmlString{referenceInput}) as input, define the code itself -(\xmlString{bateman}), indicated the sampler to use in each, and then output the results to a -\xmlNode{PointSet}. - -The three ``stats'' steps take one of the point sets as input, use the basic statistics postprocessor as the -model, and output to a RAVEN XML output file. Note that we can re-use a single basic statistics -postprocessor, rather than create separate entities. - -In the \xmlString{readStats} step, the three RAVEN XML output files are inputs to the RavenOutput -postprocessor model, and result in a \xmlNode{PointSet} that has the statistics from each file in a single -data object. - -In the \xmlString{plot} step, the statistics data object is passed to two \xmlNode{OutStream} plotters. We -include the \xmlAttr{pauseAtEnd} attribute to ensure plots printed to screen are retained. - -With the \xmlNode{Steps} defined, we now define all the entities used throughout the calculation. We won't go -over them in great detail, but will point out a few notable features: -\begin{itemize} - \item Data Objects: - \begin{itemize} - \item There are three data objects for storing the samples from the samplers, and one to hold the - statistics. Note that while the output of the Bateman code is histories, since we only define - \xmlNode{PointSets} the code keeps only the final value for output values. - \end{itemize} - \item Files: - \begin{itemize} - \item In order for RAVEN to keep track of the XML files in which the statistics will be written, we - define them in the \xmlNode{Files} block. Note that these files need not exist before the RAVEN run - starts; they will be created in the run. - \item Additionally, we define the input template for the code itself, used in all the sampling steps. - \end{itemize} - \item Models: - \begin{itemize} - \item In the Models we define the interface to the Bateman code as well as the two postprocessors. - \item Note that in the \xmlNode{BasicStatistics} we request the expectedValue, variance, and number of samples as - metrics for all four outputs of the Bateman model, even though we only need the expectedValue and - variance for output C. This demonstrates the ability of the RavenOutput postprocessor to be selective - about the values it retains. - \item In the \xmlNode{RavenOutput} postprocessor, there are two operation modes, \xmlNode{dynamic} and - non-dynamic. Because the \xmlNode{dynamic} is not specified, we are operating in static mode (we look - at dynamic mode in the next section). This means we specify several files to load from. Ultimately - we want a point set that contains the following information: - \begin{center} - \begin{tabular}{c | c | c } - ID & mean & variance \\ \hline - 1 & MC mean & MC variance \\ - 2 & Grid mean & Grid variance \\ - 3 & LHS mean & LHS variance - \end{tabular} - \end{center} - To get these values, we assign a float file ID to each file in the \xmlAttr{ID} attribute. These - values can be whatever we want; they'll be used to keep track of which set of outputs come from which - sampling strategy. Within the \xmlNode{Files} node, we instruct the RAVEN XML output reader how to - read in values using the \xmlNode{output} nodes. The \xmlAttr{name} attribute determines what column - (or variable) in the PointSet a value will go in, and the text of the node gives a path in the XML to - find the value. For example, \xmlString{C|variance} instructs the postprocessor to look in the RAVEN - XML output file under the node \xmlNode{C} for a node named \xmlNode{expectedValue}, and use its - value. - - Over the process of reading all the files, entries for both the mean and variance are collected, - giving us a data set of realizations where the input space is the file ID and the output space are the - desired metrics. Note that it is not necessary to retain the original name of the statistics; we - chose the shorter name ``mean'' to read in the ``expectedValue'' entries. - \end{itemize} - \item Samplers and Distributions: - \begin{itemize} - \item The Distributions are unchanged from previous examples. - \item In defining the three samplers, we only note that we specified the number of samples so that they - were equivalent across all three samplers in an effort to make a fair comparison. - \end{itemize} - \item OutStreams: - \begin{itemize} - \item We define our two plots here, using fairly standard syntax. Note that when these plots are - generated, the File IDs are on the x-axis, while the statistics metric (mean or variance) values are - on the y-axis. The File IDs can be traced back to the IDs given in the definition of the - \xmlNode{RavenOutput} postprocessor. - \end{itemize} -\end{itemize} -When this code is run, the following plots are produced: -\begin{figure}[h!] - \centering - \includegraphics[width=0.7\linewidth]{../../tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter} - \caption{Sampler Comparison Plot, Means} -\end{figure} -\begin{figure}[h!] - \centering - \includegraphics[width=0.7\linewidth]{../../tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter} - \caption{Sampler Comparison Plot, Variance} -\end{figure} - - -%%%%%%%%%%%%%%%%%%% OLD -%In order to accomplish these tasks, the following RAVEN \textbf{Entities} need to be defined (these are -%similar to the statistics example in section \ref{sec:SAraven}): -%\begin{enumerate} -% \item \textbf{\textit{RunInfo}}: -% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{RunInfo} -% As shown in the other examples, the \textit{RunInfo} \textbf{Entity} is intended to set up the desired -% analysis . In this specific case, two steps (\xmlNode{Sequence}) are sequentially run -% using forty processors (\xmlNode{batchSize}). -% \\In the first step, the original physical model is sampled. The obtained results are analyzed with the -% Statistical Post-Processor. -% \item \textbf{\textit{Files}}: -% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Files} -% Since the driven code uses a single input file, in this section the original input is placed. As detailed in the user manual -% the attribute \xmlAttr{name} represents the alias that is going to be -% used in all the other input blocks in order to refer to this file. -% \\In addition, the output file of the \textit{PostProcess} \textbf{Step} is -% here defined (XML format). -% \item \textbf{\textit{Models}}: -% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Models} -% The goal of this example is to show how the -% principal statistical FOMs can be computed through RAVEN. -% \\Indeed, in addition to the previously explained Code -% model, a Post-Processor model (BasicStatistics) is here specified. -%Note that the post-process step is -%performed on all the variables with respect to the parameters used in this example ( $A,\, B,\, C \, and \, D$ -%with respect to $sigma-A,\,sigma-B,\, decay-A,$ and $decay-B$). -% \item \textbf{\textit{Distributions}}: -% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Distributions} -% In the Distributions XML section, the stochastic models for the -% uncertainties are reported. In -% this case 2 distributions are defined: -% \begin{itemize} -% \item $sigma \sim \mathbb{U}(0,1000)$, used to model the uncertainties -% associated with the Model \textit{sigma-A} and \textit{sigma-B} -% \item $decayConstant \sim \mathbb{U}(1e-8,1e-7)$, used to -% model the uncertainties -% associated with the Model \textit{decay-A} and \textit{decay-B}. -% \end{itemize} -% \item \textbf{\textit{Samplers}}: -% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Samplers} -% In order to obtained the data-set through which the statistical FOMs need to be computed, a \textit{MonteCarlo} sampling approach is here employed. -% \item \textbf{\textit{DataObjects}}: -% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{DataObjects} -% Int this block, two \textit{DataObjects} are defined: -% 1) PointSet named ``samplesMC'' used to collect the final outcomes of -% the code, -% 2) HistorySet named ``histories'' in which the full time responses of the -% variables $A,B,C,D$ are going to be stored. -% -% \item \textbf{\textit{Steps}}: -% \xmlExample{framework/user_guide/StatisticalAnalysis/statisticalAnalysis.xml}{Steps} -% -% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% Finally, all the previously defined \textbf{Entities} can be combined in -% the \xmlNode{Steps} block. As inferable, -% 2 \xmlNode{Steps} have been inputted: -% \begin{itemize} -% \item \xmlNode{MultiRun} named ``sampleMC'', used to run the -% multiple -% instances of the driven code and -% collect the outputs in the two \textit{DataObjects}. As it can be -% seen, the \xmlNode{Sampler} is inputted to communicate to the -% \textit{Step} that the driven code needs to -% be perturbed through the Grid sampling strategy. -% \item \xmlNode{PostProcess} named ``statisticalAnalysisMC'', used -% compute all the statistical moments and FOMs based on the -% data obtained through the sampling strategy. As it can be noticed, -% the \xmlNode{Output} of the ``sampleMC'' \textit{Step} is the -% \xmlNode{Input} of the ``statisticalAnalysisMC'' \textit{Step}. -% \end{itemize} -%\end{enumerate} -% -%Tables \ref{ScalarMoments}-\ref{SensitivityComputed} show all the results of the \textit{PostProcess} -%step. -% -% -%\begin{landscape} -%\begin{table}[h!] -%\centering -%\caption{Computed Moments and Cumulants.} -%\label{ScalarMoments} -%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} -%\hline -%{\ul \textit{\textbf{Computed Quantities}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline -%\textit{expected value} & 5.97E-02 & 3.97E-01 & 9.82E-01 & 1.50E+00 & 5.57E-08 & 5.61E-08 & 5.07E+02 & 4.73E+02 \\ \hline -%\textit{median} & 2.45E-02 & 3.06E-01 & 9.89E-01 & 1.54E+00 & 5.73E-08 & 5.62E-08 & 5.11E+02 & 4.70E+02 \\ \hline -%\textit{variance} & 8.19E-03 & 6.00E-02 & 1.19E-02 & 1.49E-02 & 7.00E-16 & 6.83E-16 & 8.52E+04 & 8.64E+04 \\ \hline -%\textit{sigma} & 9.05E-02 & 2.45E-01 & 1.09E-01 & 1.22E-01 & 2.64E-08 & 2.61E-08 & 2.92E+02 & 2.94E+02 \\ \hline -%\textit{variation coefficient} & 1.52E+00 & 6.17E-01 & 1.11E-01 & 8.15E-02 & 4.75E-01 & 4.66E-01 & 5.75E-01 & 6.21E-01 \\ \hline -%\textit{skewness} & 2.91E+00 & 9.88E-01 & -1.49E-01 & -9.64E-01 & -6.25E-02 & -5.75E-02 & -2.18E-02 & 7.62E-02 \\ \hline -%\textit{kurtosis} & 9.56E+00 & -1.12E-01 & -6.98E-01 & -1.50E-01 & -1.24E+00 & -1.21E+00 & -1.21E+00 & -1.20E+00 \\ \hline -%\textit{percentile 5\%} & 2.87E-03 & 1.48E-01 & 7.89E-01 & 1.24E+00 & 1.42E-08 & 1.45E-08 & 5.08E+01 & 2.97E+01 \\ \hline -%\textit{percentile 95\%} & 2.51E-01 & 9.19E-01 & 1.16E+00 & 1.63E+00 & 9.54E-08 & 9.48E-08 & 9.59E+02 & 9.49E+02 \\ \hline -%\end{tabular} -%\end{table} -%\begin{table}[h!] -%\centering -%\caption{Covariance matrix.} -%\label{covarianceComputed} -%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} -%\hline -%{\ul \textit{\textbf{Covariance}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline -%\textbf{A} & 8.19E-03 & -1.11E-03 & -3.09E-03 & -1.13E-04 & -1.28E-09 & 5.14E-11 & -1.49E+01 & -3.74E-01 \\ \hline -%\textbf{B} & -1.11E-03 & 6.00E-02 & 2.26E-03 & -2.96E-02 & -7.80E-11 & -6.02E-09 & 7.00E+00 & -1.47E+00 \\ \hline -%\textbf{C} & -3.09E-03 & 2.26E-03 & 1.19E-02 & 7.15E-04 & -1.44E-09 & -4.11E-12 & 2.63E+01 & 3.19E-01 \\ \hline -%\textbf{D} & -1.13E-04 & -2.96E-02 & 7.15E-04 & 1.49E-02 & -1.21E-10 & 3.01E-09 & 1.12E+00 & 8.01E-01 \\ \hline -%\textbf{decay-A} & -1.28E-09 & -7.80E-11 & -1.44E-09 & -1.21E-10 & 7.00E-16 & -1.73E-17 & -1.26E-07 & 2.07E-07 \\ \hline -%\textbf{decay-B} & 5.14E-11 & -6.02E-09 & -4.11E-12 & 3.01E-09 & -1.73E-17 & 6.83E-16 & -1.86E-07 & 3.91E-08 \\ \hline -%\textbf{sigma-A} & -1.49E+01 & 7.00E+00 & 2.63E+01 & 1.12E+00 & -1.26E-07 & -1.86E-07 & 8.52E+04 & 1.79E+03 \\ \hline -%\textbf{sigma-B} & -3.74E-01 & -1.47E+00 & 3.19E-01 & 8.01E-01 & 2.07E-07 & 3.91E-08 & 1.79E+03 & 8.64E+04 \\ \hline -%\end{tabular} -%\end{table} -%\begin{table}[h!] -%\centering -%\caption{Correlation matrix.} -%\label{pearsonComputed} -%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} -%\hline -%{\ul \textit{\textbf{Correlation}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline -%\textbf{A} & 1.00E+00 & -5.02E-02 & -3.13E-01 & -1.03E-02 & -5.35E-01 & 2.17E-02 & -5.63E-01 & -1.40E-02 \\ \hline -%\textbf{B} & -5.02E-02 & 1.00E+00 & 8.47E-02 & -9.90E-01 & -1.20E-02 & -9.41E-01 & 9.80E-02 & -2.04E-02 \\ \hline -%\textbf{C} & -3.13E-01 & 8.47E-02 & 1.00E+00 & 5.37E-02 & -4.98E-01 & -1.44E-03 & 8.25E-01 & 9.96E-03 \\ \hline -%\textbf{D} & -1.03E-02 & -9.90E-01 & 5.37E-02 & 1.00E+00 & -3.75E-02 & 9.43E-01 & 3.14E-02 & 2.23E-02 \\ \hline -%\textbf{decay-A} & -5.35E-01 & -1.20E-02 & -4.98E-01 & -3.75E-02 & 1.00E+00 & -2.50E-02 & -1.64E-02 & 2.67E-02 \\ \hline -%\textbf{decay-B} & 2.17E-02 & -9.41E-01 & -1.44E-03 & 9.43E-01 & -2.50E-02 & 1.00E+00 & -2.44E-02 & 5.08E-03 \\ \hline -%\textbf{sigma-A} & -5.63E-01 & 9.80E-02 & 8.25E-01 & 3.14E-02 & -1.64E-02 & -2.44E-02 & 1.00E+00 & 2.08E-02 \\ \hline -%\textbf{sigma-B} & -1.40E-02 & -2.04E-02 & 9.96E-03 & 2.23E-02 & 2.67E-02 & 5.08E-03 & 2.08E-02 & 1.00E+00 \\ \hline -%\end{tabular} -%\end{table} -%\begin{table}[h!] -%\centering -%\caption{Variance Dependent Sensitivity matrix.} -%\label{VarDepSensitivityComputed} -%\begin{tabular}{|c|c|c|c|c|c|c|c|c|} -%\hline -%{\ul \textit{\textbf{Variance Sensitivity}}} & \textbf{A} & \textbf{B} & \textbf{C} & \textbf{D} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline -%\textbf{A} & 1.00E+00 & -1.36E-01 & -3.77E-01 & -1.38E-02 & -1.56E-07 & 6.27E-09 & -1.82E+03 & -4.56E+01 \\ \hline -%\textbf{B} & -1.86E-02 & 1.00E+00 & 3.77E-02 & -4.94E-01 & -1.30E-09 & -1.00E-07 & 1.17E+02 & -2.45E+01 \\ \hline -%\textbf{C} & -2.60E-01 & 1.90E-01 & 1.00E+00 & 6.01E-02 & -1.21E-07 & -3.46E-10 & 2.21E+03 & 2.68E+01 \\ \hline -%\textbf{D} & -7.60E-03 & -1.99E+00 & 4.80E-02 & 1.00E+00 & -8.11E-09 & 2.02E-07 & 7.51E+01 & 5.37E+01 \\ \hline -%\textbf{decay-A} & -1.83E+06 & -1.11E+05 & -2.05E+06 & -1.73E+05 & 1.00E+00 & -2.47E-02 & -1.81E+08 & 2.96E+08 \\ \hline -%\textbf{decay-B} & 7.52E+04 & -8.82E+06 & -6.02E+03 & 4.40E+06 & -2.53E-02 & 1.00E+00 & -2.72E+08 & 5.72E+07 \\ \hline -%\textbf{sigma-A} & -1.75E-04 & 8.22E-05 & 3.08E-04 & 1.32E-05 & -1.48E-12 & -2.19E-12 & 1.00E+00 & 2.10E-02 \\ \hline -%\textbf{sigma-B} & -4.33E-06 & -1.70E-05 & 3.69E-06 & 9.27E-06 & 2.40E-12 & 4.52E-13 & 2.07E-02 & 1.00E+00 \\ \hline -%\end{tabular} -%\end{table} -%\begin{table}[h!] -%\centering -%\caption{Sensitivity matrix.} -%\label{SensitivityComputed} -%\begin{tabular}{|c|c|c|c|c|} -%\hline -%{\ul \textit{\textbf{Sensitivity (I/O)}}} & \textbf{decay-A} & \textbf{decay-B} & \textbf{sigma-A} & \textbf{sigma-B} \\ \hline -%\textbf{A} & 3.83E-06 & -1.78E-04 & -2.07E+04 & -1.86E+06 \\ \hline -%\textbf{B} & -1.36E-05 & 6.28E-05 & -8.80E+06 & -3.14E+05 \\ \hline -%\textbf{C} & 2.17E-06 & 3.05E-04 & 2.64E+04 & -2.00E+06 \\ \hline -%\textbf{D} & 6.96E-06 & 2.25E-05 & 4.40E+06 & -6.19E+04 \\ \hline -%\end{tabular} -%\end{table} -%\end{landscape} - diff --git a/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml b/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml deleted file mode 100644 index 9ff94042ec..0000000000 --- a/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers.xml +++ /dev/null @@ -1,233 +0,0 @@ - - - - - sampleMC, sampleGrid, sampleLHS, - statsMC, statsGrid, statsLHS, - readStats, plot - - comparingSamplers - 8 - - - framework/user_guide/StatisticalAnalysis/comparingStatistics - talbpw - 2017-05-11 - PostProcessors.BasicStatistics,PostProcessors.RavenOutput - - This test is an example for the user guide to draw from. - It compares the basic statistics from three sampling strategies in a plot. - - - - - referenceInput.xml - output_stat_analysis_mc.xml - output_stat_analysis_grid.xml - output_stat_analysis_lhs.xml - - - - - ../physicalCode/analyticalbateman/AnalyticalDplMain.py - - - - - - A,B,C,D - A,B,C,D - A,B,C,D - - - - C|expectedValue - C|variance - - - C|expectedValue - C|variance - - - C|expectedValue - C|variance - - - - - - - 0 - 1000 - - - 0.00000001 - 0.0000001 - - - - - - - 1296 - 42 - - - sigma - - - decayConstant - - - sigma - - - decayConstant - - - - - sigma - 0 1 - - - decayConstant - 0 1 - - - sigma - 0 1 - - - decayConstant - 0 1 - - - - - 42 - - - sigma - 0 1 - - - decayConstant - 0 1 - - - sigma - 0 1 - - - decayConstant - 0 1 - - - - - - - referenceInput.xml - bateman - mc - samplesMC - - - referenceInput.xml - bateman - grid - samplesGrid - - - referenceInput.xml - bateman - lhs - samplesLHS - - - samplesMC - statsPP - statsfileMC - - - samplesGrid - statsPP - statsfileGrid - - - samplesLHS - statsPP - statsfileLHS - - - statsfileMC - statsfileGrid - statsfileLHS - statsReader - all_stats - - - all_stats - meanPlotter - varPlotter - - - - - - sigma-A,sigma-B,decay-A,decay-B - A,B,C,D - - - sigma-A,sigma-B,decay-A,decay-B - A,B,C,D - - - sigma-A,sigma-B,decay-A,decay-B - A,B,C,D - - - ID - mean,variance - - - - - - - screen,pdf - - <text>Means</text> - - - - - scatter - all_stats|Input|ID - all_stats|Output|mean - - Sampler Number - Value - - - - - screen,pdf - - <text>Variances</text> - - - - - scatter - all_stats|Input|ID - all_stats|Output|variance - - Sampler Number - Value - - - - - diff --git a/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter.pdf b/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/meanPlotter_scatter.pdf deleted file mode 100644 index 2a74b3907ab1310bdc6aab19ec406a39011ab5e8..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 11430 zcmb_?2{@Er)Hp>L%1%o4B3qW(hq3S3_pML{gULR|mJ}6QC<#TfL_(V-g|ykSChgKD zN_JAwBHw)nMSQ={^MC#`&wcKF&wKB==bm%!Ip@CTiks``$)V+O2=Pm=po^6VG=zfu zTy`N;R3M~vm_HdpYLjRrZ$A$RX-@JW2SOOo!4gtaLy&#l;Esy(2H2aq>>|6;AO(6K zFn~rQQ+**k{D?rBQ~g}6$TY|SpwiI;=*gip2x;I0&}z-r+OxF*_#DJ`Go?~Vf1(f0))e2U>Y@jef?;G z01n!gF@p$3_s7&t4Hw$41?NHotk>rAtHx498 z#A)-J#6ProvO=6Btj?$U_~&q2bLCH^%&xB#`(4v{ET4*p&1=GQP6}oeo*3QN_h_X` zu|ZNkH1` za;w}gYEhb+cXc(jx)HL%$7Gm)>2V$m*70Cw<74LbU>1ts!nH1RJ>sQmLW!{^iv8x+ zROOdBEtU=SOh?vx`K1>}-BJszVH38=%J(tOTKjBog`MS!O}vm&t?HE@Z*LACQqTN3 z^14`hovCOJZlmGZ7DNAU(h}@HLYkd-ls(FNE+p#1GkAzHl5!!W@*zuUp4n|3_450D zsl50*J|jY-f%mcda5BFKkN!#yPtSA+xwkKH>cg4*SkA$(RSiFmPO1M|@xZ5U#52(4 zR~>dtA%Uy5r=FhNma}GouCB0N6Y!WhQ-Oxh7KYEVE60A_>MW_`# zd3iC2k9sKx=@>^1wL{6x%)HHvr7f~8QL~cl3Hs*St^eA0IO0ND2-nO}<|rJBB~dR%Bjv#~EL)XBmHXFsI0OXHe9~-BuUBPOxk-3l0uYZEY6%vY0N+ ze8c%)48Z^sJ6}3@^XApsXx5t^2!VXQ6;W;(wzq<&y?h=jlenj~>>H9;!wMQ7NcS13 zDH(HKm%mdi%6>UWe6T4E%8(h!PAw^JfAQYgf1SjRNC5@sPn7s|cv{!~Jo#j;s5zowFzV9NM@dT-6JTu*l5cvHsu@9WctMwLr_ zyCio7i?&nwF;7Zy_t&!W>TXmC)6MgLU-~TMDURofJG*HL;}ui&xHSyUqC2)l4qnkA z8a+C8KGU!2{sgB6gVnVqSTo-#{ugV2g3sp+7S@%>8a{1#hIL6R2A4n#JJaQC2n=rF zj*zn;Ca%pO;mgz!w^&%Mmt z%`q~90gbiHRzWWsQftie;@F2A+g3MJa6coj%{`M`A9qm9M8r}*yI-OJtC_%*bI~Fv zUbt>=U$fX*l8x9dj-QT3ozDY#AFRVE3tUtop5JPh-rQKnF2gca)p}Rgdfi18E*}3o zo!V3BgFOmw9#xdqeVN=5SW<=Jw;%ghwm!k6Y*GRFep0o2;QOzY`R5%3mS7<4{+Gi* zGy%InG$skkYidv&+98?y1zDl~?zS*X+OCm1wp5<11NT^(5~P9xS>4}diB!G0dsJ=% z`TmWpOJ?1Zk{>D$M?`)*Hx!7)_CQFQZ+nJ{B_Szg}GH0i!GFR+_V-x$?DqvdvJ zd(%WfKc7eVG=utpB6f+j!cow2)~dKr3ssNZ|TOAILccNc?$RilEB9o!{n`m$!lih{nmKC`sWR{NzYtC;f9`%_cBUoMJBy0 zUwU&*#&2=4GSyn5tQo8|~hYQJFG5+uQa;&MpBVi#^Nyrog1 zgC=X|9ji|(uI&2V8kU~+wC8sx>)@rVC;^@X0mX(|BMysrYTc%={bNc|cLqLCeyP2Q zezO09=+xaZ%KDM}X?TkwoQrge+P5hwk^5HNZ@E5I=<~X}VW^PMp`T@H`<|UKCjMHL zO9}iTWw4t|?#1hM!F#qPXg)f*E@iMMHb}0hlGWt&dM{BSVi^YCc0Z#Fld)Dz^QkOk zkXdfVAE%GqsO%vZme3ViD2Uv9EB-m=c)=+fX3TZAkIu@eY!{?Nx$b0bt~h+tvtR0% zdwp7GbWEb?#B0^`^e>fn(sh`3uyx5k<2fd@y9pzyyxK5p*9WH4q+8#j1JVeF>W9;dPVeYuvs_#zT+>RA36%vn+Pf5Mh z-^BO4*U=>PvQ*cm!k)wY&nKsUy=iMxPf&;`UP3g>4Fxc`h1zUlannZ!%`xzn&+#3@ zfLB-syQR}NKkY`=glZ@oDSbg);9I>BM;D6&=k1;oHro3(s-p+7vZ{?ePt-qFG+OiB z<}W;R+Fa13Ys!_+a8)4%aDX37qpKKLZKDr&UciQ`YBbMS zuMIKblfA;dEeJF=rDB%CbT-|#1$te)KNF)BNCanJ3sGRZ*g?L zOwuzp+PP=7l`Zcs_I?Xi&_;fKmtPqg!7*S|RQ>6Exa^)QTANzF7%PVuh}{~^p$)g|JF#&jlGX7Xoa+xQeazm(&aCHS}8@I_&vv&NR9EI=s5;DQarG6#wo zMFZy~#*WR>kUzz255+b=e8|;=A)qZ$I2?0{acg|G3o2e?n^A%$(R_;NNMSIu=0Pq& z&2nQJEDE<^^Q$H*C&Fep{Oo?ghPcU-J>zbY$rHba)G&>w?WC3z|kMS^X* zy{pSb`9sKyjMkRBWjaXSX6d$Unc7#n-%C>CD@&B>3Tsej9lqqJAB{0Nnby0-NazN8 zMorn_-js-1P6b>(OHjN~_3GD8J+~oR{ez~JZ*A=e!K(NNQpRxqx1xLdaT+tj+HuI{=7r$+nO8eN~$$?8T6J)s=Nr=Z6@z4*viG}R$)D9 zYB4zU5PN&jkWH26dYAohgmPMO-?}r3gL_Bxo;Af?@f6wHU?({s?)=J1LYK$Q^t!$8 z(We>1*;09dF@Zu*f5(v+%a1O07x!@}`^h3ZWE+xN*fDw+OfnQh8}`PheF;nM4V_~2 z9uHJr%y{gwgEs8KZKP{qKxGo)xH%#7C90rWW;DaJwP1YOOBQQjD&XverbI=_g&r1p zcDE%8S+U3D6K;+Tj_s^6NB zoUHU!3n%eHgj-b%2vIihK(6#<%MZ7u93tiWldqd9$0ufq@pbtniFNhy8wQ(rue6qRrWiA z(eJUfZGzF74-V#Pq<cG&Xir7mxgeoxd% z6I_?9x^|*VOIDJ_1gh-Vmw=KJTZ0}pcBB*KQagq(3>oj?w3kS=_#l0@7o#RwFz#p? zFOoN4A$Q3^mp8xz+7@ zUzza5v+*mAi1M}cZ24jz_4KUt`<61DvaiJ$>Z`sD2i~vly^?NSQf^VNRH($SnAh9% zqj30Fxop#Iu~-bV_cP)eg8zr>q&VK3R@;!+pE?N_P8Q_64^Hr&JaDr9cGFY-a#O<3 ze(|*|`}bzE8Gkm(3X0L_{wTeh%Jyw6;O4kS(Tys$TyNH%2Cld@nm0|a>Jxy2E@=##IF`4{&T4X-svcK911tbu4Ort(n*MYtz64 zr}+{0LrZAka%0zjln{%#0ZyPrI7Xfr0SUxM_f_y|IEBkrUeH*UrDa8Xia$a$V--VY z1V`BtlP))iL=(^pEVXWe5l;=op*@kVwRLpssy!LBO2d&mmqkoUBQF$dtO}82Dkx*- zZD&UM7*(>XIj}vb5B)^&#q;zq1x4ECc6%mSMpV>Ml4)a}gg_bg%e))h*X&Np5z69H z;(s%Au7{d8{NVdm|F1j0ZY>^8Z>_!S_F>II_pj2!d(LdIaC+^w?~$wTGp~L8-wKPa zT{{2aVCMrTY!=gSf$-22b3vfe!6jC@+;|j&B`jE}iMf%ljtEC*;1S1n2Af3~A80?d z>HXbVTw^`7C1&kDea^lTn~eijEqQ!gu;%gXXmxac5mV z@;s}n-mLNgC$?Vlx$LIn%4X}`qw8zqyU)rCIcEwm;vJG(qli9A3L2FO>uwa5zHu|> z5bf7$WjQIIexd!;A!0_pvtfutS;BfpwWrxD1Ge8nRJa~J=~pYfE&DF%p>e>oos7@& z8%wq|Kn5`)4@`fPW9hgWK}-VNeC z8Sy=Y;N8^Ex6-z8HWujX^FOXcnC3L_8NEoEOgobH=t=YF+fx%l`=%MO@Aoh+A(Z9D z%4qyQPClW68+@733AE8<<0)e8FTUR%GF>3}-SCO+)nQP*e29oEzDxu}=od^9##^+>;+unw#qOjeFW*PpVeq`z1$B zK22t-Tr2qGkad+kDMF)6@vMP4I^wEJR8D&EKGF2G@6?30rb>$ixcDLlCxjT>Z`+mE zcQe0cR0vyQmCKE2vEWP9{I4u{i<<#;D2@R5;cD4+^JS(tlw63%mPxb+fG<_Zl_t9Jx11P)tP{gzaR`<~lC;drezM zLvf<^X1>o6Wp@vWc}H?*n+=I^f$WIx#Bi#mnuN`f5e7XG;fO*zOqJk?j$JXW(?U)i zTM(;kwr$GP^Orbapg8W9gLrIN_V~`vSKZ?*uCBWymk`9VgZV|LX5y9C8XN-=2q!2q zxHY|(xnS?`+nA)C2<>aON79Z7BvZbVMQr3w%HJ6>@HxEx^F$-!FmIxr_{;MgC(OUA zL|fnSXs){bES(!NN=BSKnRs8E{Wr&{H}b;0u3qZL#X?kfJKMw^)nQ9v0ZEIhM8ca>RtawLR26 zBy8Xx7jJ$dGC1m`J_oT>;ATGZ$F`)?Y_y@HqQmFz@))$FyErNR;8df@=HDT7H(xzw z_(NaD%lKtTjsc}b^yHOOx_%0N5vt`HEV>lD1OcG?!vC z;YMV`HEa89+t)eTW6@H)oD6dBqL{mG3r~*;=HGi>6iuxOcU$3X_{3MEUb|gLf14&Z z>E{XlPr2q-y}L%FAiF1*qO)l(hgPRBNZ-D=`ta%ej;z>&$%%y<9*xE2jyl|PI!rQm zaOh+Tb$`g!@#aK4H+HC9C}116m%3_Sjs>R7LWtAEW6#)YjOBPcK>MiSb(6M7o0g>W z=8p!ad;P__GCDQc(O&|trkHt8oR+pdG0Lpm#JKX-)SJZkw(xhdagLpX#*(A$lncVV zmv3)$nL42EZFI)*zD%qxUw)Mo>)z*@ya9Xc>n7ZCbV9B%J`>}8>M72}!BhQU2=zI% zJ`DJeYahSpc;@JAd~%FiRjQ=!Lpp!agdBzSlJ^)}ZH zPeL+&{`kcx&yX#&gh2n#lZaK23yq$J)$^nJ{7D9a)7)%!b~fdTKx)n8C*{zJI7kkT zY0wx&MTm$Yg4E=EbJ>l-p{!5@yiLOgi~?arDfO*7)FB7DQYrp4KPrT#XJ;X#Dai+9E43)Jnd5>jnM$$(DONd4KOd4W0tshpO-M8< zg`P!~N1@=Qga4lv$e?PG0?G8O?*ER)j^al141~}#d0*ro_Q1JnEXebElBjT=8T5cd znLkO`8LWB~Y9I~n0-yWapYjKFN7phJUnR# zq%Q^ZRRH2N0UU%BK}Rb{5#Y3ihyX}}kS-7!unI{A6~ONeA%Pb7f^0CLz#mjN2niC# zfe19nJs^W1C&a8?{2?C59%)RPUH0seY%WJP|1bbc6i#{5b%!pk|6mTFl1BEi1ssNR z)gFO>Aqb?sK7CCP9(=N!r2=OAJXN8i{Xg`?gcM4bF%U5htWlffKZ{lYeC$9PfV&73 zS503JZ!$c&<_yiSmV-sQ(`Y@&Aeki&lhiKzuj?3dmLwB47y!A|3@P zfaD$qdM8Fe(f&wHUSE2xvSpg&x2q@5oYxq4D6p+_d0OMht zL`5Q~XJBwmUkp|O0oO1Q@S@;C0Gkyaiw6sz!2$Y$dFgE|4lqKIUhoP;dQDUS^a3~m zjR6asnHLA<#bW?tfMbK-;X$nkm<8ALf`R)H=uCiXcz(LjUQEO1VjQ6nk%p(%;*5D z2MY_G*FY~|otRhX`T*<1T%qgAtWLoCFu%Zb(VqbhSU2VhedQ=T;5(od)`2AoQ4z=w z)(hZa=9f90n5$u?0J$URHQb(8=(3)95BK>~V7-8)3fE45jKlmB?u;p>UrVt3W5xkD z>5c<7;%GP)2CwL5ot_~F4>N`z$Qryt(9Jw%=2Fy*F$dhBk1>HTbDj|bdO(L+L6}*$ zM0W=m`b8-OF!q?)K_tNOnXWDn{o^i}j||jr2Id1h9y%g#py@L$UkEIjPKiI*RWmSP zk7uW$Lcl7}M+8AYDe0{lpM;&AXU1mJBg^^My5?*se0fR_DB=v}zeaYx>(Io^+_BOG z7cS=9&!4+nNLLa7!(K=eJ=YCb?wPw~f3JAWb(?3$=fxa<=bb0MzrgtiWH|1S^P%_# z1wy!4wwZ%nJ;S%ZL+F72%nkktEfz#xXu!8w;unQpSa}e4p$G~@c(=$cc>3QT7Di`2 z322a)X4MTb?PA}xCow+Lbm)pRT&l?P`uhA%ezB=&LJ}Dg~3<|A)!lTh>U=Gm)ITS$> zg_4w}-xqWXas~I{qLEQ(y+WyEcLWMtKKlbc5DzwyJM;$zhmr6P@|}gDQ5aye7Q#?O z*pDuP0bBboOc6wQ|H6nMDYgg(TMYR3XI{`IEQVo$llr$G1_X-#zyNa>(? - - 300 - 1 1 1 - 1e14 1e14 1e14 - 0 100 200 400 - 100 100 100 - - - N1 - 1.0 - $RAVEN-decay-A|10$ - $RAVEN-sigma-A|10$ - 230 - - - N2 - 1.0 - $RAVEN-decay-B:0.000000006$ - $RAVEN-sigma-B:5$ - 200 - - - N3 - 1.0 - $RAVEN-decay-C:0.000000008$ - $RAVEN-sigma-C:3$ - 150 - - - N4 - 1.0 - $RAVEN-decay-D:0.000000009$ - $RAVEN-sigma-D:1$ - 100 - - - - diff --git a/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter.pdf b/tests/framework/user_guide/StatisticalAnalysis/comparingSamplers/varPlotter_scatter.pdf deleted file mode 100644 index 482cf3a61a9623d8c1628379dc787e9bffbfd035..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 11892 zcmb_?2{=_<)UYvJ%1lai%w)LpPIc@Bkg%_75fO^GO!LNZT@$oN(=Mw2KRlTs?> zO_4H{QZ&f7&qX2M`+5H7zn*QMv(DOUuf5jVYprwE7BkV*mPO0q5MuwlfXeC-Xb1&) zI~+hLD?><&U>|1)sX?NUJiT2ZqzTE@*$=`11T$#I4urFp6O5>^WPr7?!vSYU3M5a( z0Rt!$XR;TBhi?%`6SB9XxibZ_0i-mw0eR;j3WU@>2*|20^coAjE@THGwSp+S%qb*_ zGlZKbVD3h8a`$qD2=nB%y}c+9mU{h%9W)9678kIK?CtLZ!AsFFhme}i0q%~@X1l;- z;Ld_f^78W`k)6F9gBO=NkH2FFyjJjlGQU^^($Lw-ouuv^1Xzd3<`26{UX2G?w&wk5K`L{tXk9A z(c1~g-@w_+mEs1WVa-ql??)j!lMW(+vg1+>{T`t>qCb4+RcsVGc~f>vmK3FsDf%H> zwuX-eRyVIT`B$jr-q7l5PNsX;_UU$wb{>#bo7Q}=Wr)Jwi{+g{T-q}5YDDN3hsLUH_+D|@dWlzJT6X2?0rN)yyoy*riFxOP|l&yM#)mtCw)a@RGV z4GjC}681oSugi@DJ#vvPb}U-dIdLDqT6_Cs*KEK~a`pK>ztG+{Wuggv1TF6F42zp5 za2b)qVurmwq*Mj0>63{eQI%JF%PjZf@NzY6r*7Mvd7LS?Qyn8}_b;b(20>3pgV*&i ziZXlbI(+Xg0UNkNbI$GWgmFm6mli!{^LrJ)-f^-53@|Taf$A!WvsDe z`u&I}7ZB(7yu01*S#{0n;*=(1X-43=r;aK86Lt0U*AQ%h+`Ppj*KYm1T(vI81X7H@ z_EPZ3wr1|xDay|sIpSLKA&*l-ZylPIt9M7t>F`gfc^$Lp%9j4IyDQ>sINga?ZunO` zYCh`&ZXa{3I65w9trP#k9%p?xzK3f>Jt{oMuEZ<#03Cf@w`Z$V>QnrSqRsUGrWko3 zDBf4TJa?*@;?bYlH1SA<$L2yivED+QUm$b6%DsgC$ZM1mO<5dzzZ==4?%@N^_fo0v zz+m?4mRaeS2~TgdY|!pw(+nNu8u=jmj=9Fnwaic;r)+&^YPLg+T~lcN)&td@p$yO2 zZ(J;jt~E5Ny_YbT*Ho)xRiEv7bO2S&T(o^X@0pW5O-!1OPvR$Y1^8y_iOO-oG53Nv zY$99LZXWMwd_Vf7O~Lfe9?iz)%>`BBC=?g(nX7^nk96L`TrvBqjUBHv1=o8ZZ^ajN zz7CrENNMKoH@qEnCo;?{_spiTVgy6p$fMb}B&D26@+bZ+sz#S!Z?v#47W@)*zrY(y zUID%6tI)7xTFYj`5n&Jad!u0qRVFAC6b{9ds2#1EdL~Bwq!7Oj&7F>+qbzU3_pDRh zbY6k!lK64y^)$`z){C#g%u+A;U%OlZwQORo*lX(>P|TVkh4u_#;x6YTQ!0;ahV)+e z7}tkfRpi2$_miTJGvu(zM4L~p%gDFKU)45?J3E$N?zP`P;_V^xvMPPd{sKz z1*TTvYpvdgxSTJ;I5n;Yh?)Mwbzx%@%?ERoLl5Wnm-TG8(O^Uu!>4t03V%Q!HE|Wg z>tQ05o}AXO8)eJ-eBylDFO06i+2}_hwe-g-CVh`R)cx_*_Yh6l?4flp*&hlDTmQ`= zW~o2_FUetm=U=Kbyh;0d4K(vLR|H=%4_$;)j^z#iIgf)6lt`R&>eemE%)uqC_od$J z?@%;guaUcTMVR%Xzt~t?CX^%nAupriO83*(_CA}$_l5Jx+rM*<+k~eK94(YfQIEJQ z_A%qIf5^L=#sWbH1jqploEok6EC=fO46Un_&m|bqzMczfnvfmLOZ?oHv-#WR?D3CE zm0kl92Lgn<$-J1JO5DAT%-mXAm4mekeO^~S34Dy>>TzK;N~OJIv@>=CjlJ-`J>g@Q zG>Q5T<1d`^zJ70-U6sbXY8BBejn)4p8esNIB}295@FpG&Ihsw$bYrU!hGYC%wh49x zWxSa_8OXfvmbp*F{q$R*Ih2Y66Spo#N=TQ>s;bK&_dL`O>k;d_&@^bZZCdP#`i|4% znH^?#r}-**x@0f!>f)@GMKG9htIS{|c!hhqdfq7pkLN4Jskl)J83t87aWh#)C z+A7Me5~EW=z9DaUqCdeZ@t&)gxB$mX@hgT^4R2ngR7GZ_t24f}4u4Fktp6@(VEv{% za&kL~@9{Ci9DGTnOI5t1Q&Z4%qd`nmbQ70JR%i-KY1-*WFIYqhcNqqZH&)mXJ_u>% zf5XZo-@Yd)N_*VMQ55m8Je)Ivc%%H}jkl~szJ?RdXESBk4!Vh5I}{()@F2VL0;#z- zr}y(OCM}zg{8h97r#5TR0tTnBOuw?G#6)^E=JsLUL+XByK3r)(Gd=en(MEeLS_4v) z^)+cW=17(9|J1U(YT!f0#@m6n^Zsq~-^@qk;MIA!(ZaCU?)ugnOaX!}be{|GGRFi) z>32FJ6ZOLddE{tUzx9wWrQpfqkd1FCV zN^@+ah@p^~PTq)k305tEA-~KtKThy&*!y;ov!vZ32iSht>i6~fao^vBQ{pRACSKTO zmEGQYmsOf+_IhW7mc^zrWezT%TYVa{*<*w9{SWIZ?@rC^^Q*Xy;&FHtE^u-r!_*M}hoHnjSfb#fd^25hn;RH^X9(KCzwA^1?FEswBWtnj+;1_He ze0w{mf>Lfc%rrGzVE9?xG@%srfZ`JB>o&N-as&T9H@(Xz-*u^dV9FTb&s>EiI2B$C zOIYl3or0Z}Xrl)2tao&m4*cp2&dz*1_^Xe3?4R5SKCT2lg_b+|Y^HJKyW4_~PAW#+ z8Xa=~xuZX-=V+1eY(usC<`4HW@uuZC2dR!7UuPwS?wP;&m*ZWX4!4UFh71Xu_))#% z$dfDs;?H#j?tb4Tbq{gKK7F|<;P9RVwTDTYQpX05`^%QsGaIIF_7D~zR%7s8_i_d> zIU7aP9?L+w=dK#_#_C|VD!IxACkzCY@*~4;#PwoMm!$1x#MH37wO7huDUuZCxRonf z7jw;RL^9r`IkPV+I#GD~rAl`8RQ;`NO~!pJ12Rvz;sp-1VI-8+>*XF8Vn`?5_!{M# zNzl7${%G9p%H@Au#Yr#TmRFi#*5CK_L1uMuiiW&Ileo7YPoJLPjJEF}NY<~BrU*&V8*^A@s|MAup=YM=1 z>(m&N7vH3oWw9~PkVobcZzr~8+)}m9hFXg!a_O!WvF6@kE8E4O=D5^Mw)~e2(bC#a@(c1NSmX%l1cKa&wTWfA5zdTN3D^aE> zlX4mdtRI%qPeeZo#3d0{Y_6(Y8lyFIy#2ugjy4PdZHB_(7%L1qyEw~S_rQRI(oo>joX(DhYE2|}yHr=6^qm)4-zqZV>MIoQmNucz+DsK^ zpNck2${gOVFHp;x(^MTZoEmzET^?7=^kXAOAU}8@rwb;+6TcEJk|x zPxG7dzWpX-eE#@*z1N48C53E5P}+Q4gXZ0;@s+k0KkZZhx}kPcMWfmQ-Qh|J<){9K zjOD!eJ2}{$>MUlAOvlC_U~i7;v8Zs>3|LQuDrH`IzbQ*$EbN2!leXAPZbD%#RuZFP z_AkuEwYZ#&YOK9ZJW|9K5Dkur>VYc7~9lC%5sMXY*N7e~6r zE9I)jJ&YBejh9HI7(+QW6?AEFRt9{I0#lkpa(AcHN-LvsB0rr!yrJS-u@INt`RoXZ zbxirHO69{f=VGj)a{c|!)x_m(lG5mKqpppq4OSLufj5wL&vZfqkC%qIIE7T|bS9*vD1K4L zi9Z$MR1pC-FXMNtKr^t1$92+t12`Mh$KVYfXq&f zM2C*tWYcL>b^MfX#hG3H4_cpP6J;}=O%#nA9A>u`&oCX5Iy;QnAyM+#)+ltMa!dE% z`3t*$WY6}WN&O^T-Iw?Hy_J}0M%?3~gsaMPvF9d#(Z}cRp?;@Z{$>omT&>OK2+MO2 zET$H`1()M!xuW^R5pP}AA*7;hRd1HW)Tzty>#*Hvqpf5MuM*2XdcsB>rJ1ql`|Ikb*N-D{J z9gyHTb1bR(X4_-lt44$$BVrqwj)vv27<@9!^^aD4^H%B*ndR%G@3qgW<+ayY3Otzy zTR37js9iI?S8&POKuwwdT4jHpHqrP35qZ<2R6D>Y(dR^+5>dtfM&> z%2vI~q-)J)(FF9eAgh(2&(#F6X-sFU@A|lF-QgTcz21jg7ljNf!;7w{t_zf4D5+-T z?q)HEdXamJ%Z5YA`2x8d zioE^f=LgA!6ZgOE^7*p=%dRUE*`0S9oQ5_;x_ptEIGnY`)b6Fzk%x|6PdtwBel0Dp z`scz>WZ!)|Y%aq@iQxDwV~L+)ww+co-s}?9d?)VBSvdjwb9}UTo0QH7;z31u)%t`@wWXE)P9|)^ zBkG+@Npjgm-DxL?ImPyRf#TH(n{9VI&ZGC;djnDDcq+;Jj^LiW2GRoq-zWQNpA@%N z>}i2?qlNB==7%IB;)>gXrFk3KlkzyfL_f}5mr@sXzGNuz=a%PmD8`yt|MzTq5v4S@ z|J_~`dW%*x&dz)lp{+F|!~&1B^aF*g88KcI&DK7P2(zad{WO`g)u89c97T6GBV(y& zyYe8pNmDFLCZW&N<8`mLOK#)G%Vix=A9VU!6i;}xXxX-=Ho9uc45IHS-#PlBZ$@D2 zw|_cXE18;+&!z<|$4Xe*)tgUYtD3gQH*IgZVm~Te8WLn)&{QJ&^v$Wc9e4fskMi9V zace++8!i(Gx>=z0T*alY%7#7FeOC%k>alV6`|X4JqcT?a-h6|e_46vJAFv8Pj|yxK zmW#(6bk_7BxgINMzWep<)6*9i?A|py7bwtu%h=vXub0{3c^>-J@?NM2!%=wYy`Ds^ zDR1CNt8Hc7^|QJdn`y0+ojmtP8#BHHren%Fm4Z{bFBoaf2!zxytUvo^jjwhH?8=Z5@pbS*j79+= zyYn?C`I2?5pd+u&`g-5zydN|^2e)j3T+&ldBJ3<<_`=jCb9Y?~owl~HhPuZEb$w#v zOwNP{L_F7FBUbWVD@K0blbp^%89yaFalU~|w5nB{Kz_qXGpp*>Lv+k9Tk@ zAIEYfPlZZ`?bjEtmprp`!}`1<`?u!DX0P+6_3J)YbCRBa@)_x}pncTCyP1Qbs@u5_ z*?*twoeBQ+lU}eU++r2gtTj)?5|?{G6*GM=M(`oEOX{lXg^z~hoRa>Dmq%=}IxI>N zNctLdavEES*v^r}6Ig?=FJ!ydrk`$j9&mjtG$BVQoUtbFuaAC8?#^_$Vl3E-Y^k!a z&a-@(uQ3@V$<0n9`znHQ;HKc*B!BVk-ts7NQ-~9ty_y+|rVsmw z4CM5wv7)DZFQ*!NPNz#*p83eA)J98xW41prt}En~Oss9+n1RH{ZucTV?u$3KI?Nv1 z>8YP(dr$hf7EkeYN#?L#HE!R-)_13!@-+jiXrG91KXw!2VB>1MKaTno)Eo?4bk*A_ zO}Bi_tv&IaDv}jU45PM{y)?Pr#w#nl8sJJ%KqtzW=0^ zqsbFkMWO#^>TK@sK%th+wY|v)e;04T%iBUVeW65-K&sEzlV#BgI7k))(grj}K>;FS zh@h0bG#I|Y;85l$0)BVP5JsLrS4nP=5%Cj&SVd3St2sTQq1}%_{LLqNS!M=G&zjDD zj%0TqiZ>ZTQ>*C^(ui~rRF~D=Df4d!md<37IVg$Cnt2~2c_EN+b={CeA-hwn-f}1u zY|G&9_Z6sktCReksa5#@os5;c6UEIBLeJOfk-xv0%@%?zbHKTq`ANaEchkHLM8iuSc)Mjh^l3vWE@zd2FYiEtUUO-fz$=3N8BCN zyj(q<;mOtJd4}yAtaEW$i*JE85CMhy|IrZtuXM0z1qcVUh$EnYmK7iZmVhAQQ9$n~ z2m=ZQSWpRt8y@fpKZo(L;Pr;ugKZA*|EnPupJM?o++Z4u4Ua_wn}G!Yd;$Sj7lEh% z$%F5u1Yo~-Fu6S3@E8a~#3Mk12zUa72YH$T$YSJyIm;^m22kJ{?&0TH&_E$v9*l=+ z5*3J`pNGLcbun0Z1l+?+z>9(#0c=-zEFLU;o(8}L^HQ&|I3NfGYQxJDsXb91@C)Ds zGzKhieqJ1y7mopg0ig|kh6lX@5Ek518wSQBP=x^Z@cdMz!2$$40D)44g$HA(LRn-E z?x~gr&xC_-P_T$#w&A70J!n*Yz|+k)1b#thze&ws00!qtV8DvOFg(l^;2h=%HiJcV zE%rcVh($)I6maqY32beceb^3u(}%HOAk~-vCHVd~6__a$3PF9gK!8XfLW>PHgn1i) z?O=JM${N@OY!gcw)gEA*SZq{VS+EJ%9+ozkFX}y@0o%r6qplo<2XY6z!Zxr z!FB;+%+j`K6N^186rgtmwTG{lG^(!WpTju68*CS_R^i_6w{uwf?!Dl;sRudC<{dYX zCH1rw)n}vO#2-AM`gQ8r0dO<#`GKmzBLvl-W8gRrp1?jIXr6j)1Ys5;qd5eufC{5V z0L(&UvIMq1Pl5zt7AAEB@}5U^fT-Vc0a0gQeDg4OIPg#j9Rzkhf8_~*1p^oY#00QZ z>Qf(JI`db+FH`4$K#TxNqCWKpduaY@J~~koelULCcjM;g`r}L&eJOnGO&b{G{2YIt zFAX7T==sxeYJmP?82Wwc4(UjOsNZdnYM4bd5Z>kk^q<3di)c&y`I7A8&%8_2_y@T3 z6M+fX$_wIxlM&g2?q2?W5C|!-6#fjM;{0c!?st;0EC)jaDK0SoR}zMm0~r{KAWwvM zjqI|!|NUlpo^~(+4c;7?wE~R>SP!#~F%UhPZAJ8VOHJ?3vV4D9q|$nZevUS^|GazB zXSLP~^@sG2yXB;nS}Jx|)Zss0E+|b(GtbQ0W5=~|yiw+<8vBE^#9w=Vol`QrGo+?| ze1yg-c(BVSE&Oyh11IYnosGlIj!1@6B3IcPHH5D!sB_ix?iVgQOcUv9ed_OQ5SHMg zz%;!5LCOHssPFCsXGYZXl4kJn#d*EXAM*SoZQN3G(f|h!Dem50n((1aNlirz3N4Sq zqtR&K7SRM*6hQ)ol8~aF^mOuf1Sc6K50WT8p57EscL&JF$pvyCIeLJY4^XJ2KsLG* z11*P_!^%Kz6pD``68UGSoHyB3YKa8|k)2%-CLv1w+x0-P=FoB3K#+Qb1PtI;Ja7AKw9t@3=L|oEAh}+;QLpMMdM+2wtNmW zfw)pGC^Qk|3M=q1C{STo3B%$4hRLromiU*BP*^yjT0sv>0Qu2Mm^|?>UT|n|sCNY( z4)^!GIC+pmt-!+r%lZoj9}`%K2j_4rVFVECR>D?U8vz3c3;6fj4p0OF912#z6hJbx z0w#|}{k1lEJRC!pkClgG`*Ijq8OT3=!zg5uyQedmdOBR!%f%ZUgPxB}X5QYgSE0r$ ebN55wJ!_$?P9jsN+ZBx>f;SE^F)d?l#Qy-K#~?TW From 6ce1346a47ddaef8b6a985a56270d1a3d316e5da Mon Sep 17 00:00:00 2001 From: "Paul W. Talbot" Date: Tue, 12 Jun 2018 11:00:19 -0600 Subject: [PATCH 8/9] another cleanup --- doc/user_guide/statisticalAnalysisExample.tex | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/user_guide/statisticalAnalysisExample.tex b/doc/user_guide/statisticalAnalysisExample.tex index 48865b43af..44d3320899 100644 --- a/doc/user_guide/statisticalAnalysisExample.tex +++ b/doc/user_guide/statisticalAnalysisExample.tex @@ -192,4 +192,3 @@ \section{Statistical Analysis through RAVEN} \end{table} \end{landscape} -\input{statisticsToDataObject.tex} From 912a6d75922d753ca6183ab13db866483ba48d4e Mon Sep 17 00:00:00 2001 From: Andrea Alfonsi - INL Date: Fri, 20 Jul 2018 06:20:19 -0600 Subject: [PATCH 9/9] removed unused import "sys" removed unused import "sys" in file MPISimulationMode.py --- framework/CustomModes/MPISimulationMode.py | 1 - 1 file changed, 1 deletion(-) diff --git a/framework/CustomModes/MPISimulationMode.py b/framework/CustomModes/MPISimulationMode.py index 14e98637c5..88828b57a3 100644 --- a/framework/CustomModes/MPISimulationMode.py +++ b/framework/CustomModes/MPISimulationMode.py @@ -21,7 +21,6 @@ #End compatibility block for Python 3---------------------------------------------------------------- import os -import sys import math import string import Simulation