-# Version 5.2.3
+# Version 5.2.4
PEST++ Development Team
-April 2023
+May 2023
# Acknowledgements
@@ -70,7 +70,7 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI
# Table of Contents
-- [Version 5.2.3](#s1)
+- [Version 5.2.4](#s1)
- [Acknowledgements](#s2)
- [Preface](#s3)
- [License](#s4)
@@ -2693,6 +2693,8 @@ If an observation belongs to an observation group whose name begins with the str
Similarly, if an observation belongs to an observation group whose name begins with the string “l\_” or “less\_”, then this observation is a “less than” observation. No objective function penalty is incurred if the modelled value of the pertinent quantity is less than the measured value listed in the “observation data” section of the PEST control file. However, if the model-calculated value is greater than the measured value, the objective function penalty is calculated in the usual manner, that is as the squared residual times the squared weight.
+Users that wish to (very) strongly enforce inequality conditions are advised to avoid using extremely high weights for inequalities, and instead are encouraged to use the “drop_violations” functionality described later.
+
### 9.1.11 Localization
Calculating an empirical cross-covariance between large numbers of parameters and observations from a limited number of realizations is likely to result in spurious cross-correlations. Because of this, some parameters will be adjusted when they should not be adjusted. Furthermore, when large numbers of independent observations comprise a calibration dataset, a small ensemble size will almost certainly not provide enough degrees of freedom to reproduce these data. To combat these problems, users can employ localization. The term “localization” comes from ensemble Kalman filter parlance. It refers to a strategy whereby only “local” covariances inform unmeasured states in a spatially distributed filtering problem.
@@ -2705,7 +2707,7 @@ Figure 9.1. An example localization matrix.
When applying localization in a history-matching problem involving large numbers of parameters and observations, a user may wish to define a “local” neighbourhood around each observation location wherein parameters are expected to influence the simulated counterparts to observations. This, in effect, creates a series of “local” history-matching problems using subsets of adjustable parameters and observations. The number of degrees of freedom featured in each local problem can be relatively high, this allowing a small ensemble size to better reproduce large numbers of independent observations. Localization also provides protection against “spurious” (non-plausible) correlations between parameters and observations arising from the limited size of parameter ensembles. For example, standard methods of covariance calculation may suggest a correlation between a pumping rate parameter and a head that precedes it in time. Spurious correlations of this type can lead to parameter compensation and predictive bias. See Chen and Oliver (2016) for a good description of the theory and practice of localization.
-Through localization, a large-scale parameter estimation problem can be turned into a series of independent parameter estimation problems. Suppose, for example, that localization is employed in the most granular manner, so that the localization matrix contains one column for each adjustable parameter and that each column contains a single non-zero value, this pertaining to a single observation which that parameter is presumed to influence. If large numbers of parameters are being adjusted, the parameter upgrade calculation process for a given lambda will require as many truncated SVD solves as there are adjustable parameters. This can require considerable numerical effort. To overcome this problem, the localized upgrade solution process in PESTPP-IES has been multithreaded; this is possible in circumstances such as these where each local solve is independent of every other local solve. The use of multiple threads is invoked through the *ies_num_threads()* control variable. It should be noted that the optimal number of threads to use is problem-specific. Furthermore, it should not exceed the number of physical cores of the host machine on which the PESTPP-IES master instance is running. Note that pestpp-ies does not require that all adjustable parameters and all non-zero weighted observations be included in the localization matrix.. In this case, pestpp-ies implicitly treats any adjustable parameters not in the localizer as “fixed” and any non-zero weighted observations not in the localizer as zero-weighted. However, users are strongly encouraged to explicitly include all adjustable parameters and all non-zero weighted observations in the localization matrix: explicit is better than implicit…
+Through localization, a large-scale parameter estimation problem can be turned into a series of independent parameter estimation problems. Suppose, for example, that localization is employed in the most granular manner, so that the localization matrix contains one column for each adjustable parameter and that each column contains a single non-zero value, this pertaining to a single observation which that parameter is presumed to influence. If large numbers of parameters are being adjusted, the parameter upgrade calculation process for a given lambda will require as many truncated SVD solves as there are adjustable parameters. This can require considerable numerical effort. To overcome this problem, the localized upgrade solution process in PESTPP-IES has been multithreaded; this is possible in circumstances such as these where each local solve is independent of every other local solve. The use of multiple threads is invoked through the *ies_num_threads()* control variable. It should be noted that the optimal number of threads to use is problem-specific. Furthermore, it should not exceed the number of physical cores of the host machine on which the PESTPP-IES master instance is running. Note that pestpp-ies does not require that all adjustable parameters and all non-zero weighted observations be included in the localization matrix. In this case, pestpp-ies implicitly treats any adjustable parameters not in the localizer as “fixed” and any non-zero weighted observations not in the localizer as zero-weighted. However, users are strongly encouraged to explicitly include all adjustable parameters and all non-zero weighted observations in the localization matrix: explicit is better than implicit…
PESTPP-IES also supports correlation-based, automatic adaptive localization. Its algorithm is based on that of Luo et al (2018). This functionality is activated by setting the *ies_autoadaloc()* control variable to *true*. If this localization scheme is employed, a user does not need to provide a localization matrix to define which parameters can be informed by which observations. Instead, PESTPP-IES uses the parameter ensemble and resulting model-output observation ensemble to calculate a Pearson correlation coefficient between each adjustable parameter and each non-zero-weighted observation. Because the normal usage context of PESTPP-IES is that of numerically efficient parameter estimation and uncertainty quantification, it is expected that the number of realizations that constitute an ensemble will normally be considerably smaller than the number of adjustable parameters. The estimated correlation coefficient between any given parameter and any observation is therefore likely to be somewhat in error. This error decreases (eventually to zero) as the size of an ensemble rises.
@@ -2739,6 +2741,8 @@ While detecting prior-data conflict is relatively simple (and PESTPP-IES will do
With these options active, PESTPP-IES will remove observations in a prior-data conflict state from the parameter adjustment process, that is, these observations will not feature in the residual matrix used for upgrade calculations.
+Note that observations tagged with the “drop_violations” flag will not be “dropped” for prior-data conflict reasons.
+
### 9.1.14 Multi-modal solution process
The theory that underpins PEST, PESTPP-GLM and PESTPP-IES is designed to seek a single minimum of the objective function (e.g., a single peak of the likelihood function). For many data assimilation problems however, the objective function surface is not convex with a single minimum located neatly at the bottom. Instead, it maybe pitted with local minimum and/or contain a curving high-dimensional trough of nearly equal objective function minimum. In the context of deterministic parameter estimation, the goal is to seek a unique minimum of the objective function, that is seeking a minimum error variance solution. However, in the context of uncertainty quantification, and especially in the context of evaluating the likelihood of an unwanted outcome, exploring local minima and/or this high-dimensional trough is paramount.
@@ -2889,6 +2893,8 @@ Where model runs are based on random parameter realizations, the risk of occasio
PESTPP-IES provides a mechanism for detection of model run failure that extends those provided by its run manager. If the objective function associated with a particular model run is calculated to be greater than a certain threshold, PESTPP-IES deems the model run to have failed. This threshold is supplied as the value of *ies_bad_phi()* control variable. In addition the absolute filtering/rejecting of realizations provided by *ies_bad_phi*, users can also activate a relative filter with *ies_bad_phi_sigma*. This option accepts a floating-poin number that represents the number of standard deviations above the mean that a realizations phi value must be before it is considered “bad”. This fitler is adaptive in that as the realizations move toward a minimum of the objective function, the measured phi value of each realization changes. A setting of 2.0 for *ies_bad_phi_sigma* results in realizations with a measured phi exceeding the mean phi plus 2 standard deviations being removed. This obviously assumes a normal distribution of measured phi values, which is often violated. In the case where users are concerned about this normality assumption, the value of *ies_bad_phi_sigma* can be supplied as a negative value, in which case the absolute value of *ies_bad_phi_sigma* is treated at the empirical phi quantile value above which realizations are treated as “bad”. For example, an *ies_bad_phi_sigma* value of -95 indicates that realizations above the upper 95th quantile of phi should removed.
+PESTPP-IES also supports a more granular approach to identifying and removing “bad” realizations through the use of an additional column in the observation data section (this requires the version 2 control file). This column must be named “drop_violations” and it should contain entries that are either “true” or “false”. If true, then any realization that violates the observation condition will be dropped. This functionality is designed to work with inequality-type observations so that if a realization violates the inequality, it is removed from the ensemble, rather than penalizing the objective function. Note “drop_violations” also works with standard equality type observation, in which case, any realization that does not numerically equal the *obsval* quantity of the observation (to within 1.0e-7) will also be dropped. The “drop_violations” functionality respects the weight values for observations, so users can activate and de-activate the violation enforcement by simply changing the weights from non-zero to zero, respectively.
+
To forestall excessive PESTPP-IES run times incurred by occasional model failure, it is a good idea to set the *max_run_fail()* model run control variable to 1 (the default value for PESTPP‑IES), and to choose values for the *overdue_giveup_fac()* and/or *overdue_giveup_minutes()* control variables judiciously; see section 5.3. Note also that model run failure does not hurt PESTPP-IES as much as it hurts PESTPP-GLM or PEST. This is because the value of any model run undertaken by PESTPP-IES is lower than that undertaken by PEST or PESTPP-GLM. For the latter programs a failed model run during finite-difference derivatives calculation may lead to an empty column of the Jacobian matrix. In contrast, because PESTPP-IES uses an entire ensemble to fill a Jacobian matrix, a single failed model run does not result in an empty Jacobian matrix column. The outcome of model run failure is that the number of model runs employed in the averaging process through which this column is calculated is reduced by one.
### 9.2.8 Reporting
@@ -2983,7 +2989,7 @@ Table 9.4 lists PESTPP-IES control variables. All of these are optional. If a va
Note also that the number of control variables may change with time. Refer to the PEST++ web site for variables used by the latest version of PESTPP-IES.
-| Variable | Type | Role |
|---|---|---|
| ies_num_reals(50) | integer | The number of realizations to draw in order to form parameter and observation ensembles. |
| parcov() | text | The name of a file containing the prior parameter covariance matrix. This can be a parameter uncertainty file (extension .unc), a covariance matrix file (extension .cov) or a binary JCO or JCB file (extension .jco or .jcb). |
| par_sigma_range(4.0) | real | The difference between a parameter’s upper and lower bounds expressed as standard deviations. |
| ies_parameter_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing user-supplied parameter realizations comprising the initial (prior) parameter ensemble. If this keyword is omitted, PESTPP-IES generates the initial parameter ensemble itself. |
| ies_observation_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing user-supplied observation plus noise realizations comprising the observation plus noise ensemble. If this keyword is omitted, PESTPP-IES generates the observation plus noise ensemble itself. |
| ies_add_base(true) | Boolean | If set to true, instructs PESTPP-IES to include a “realization” in the initial parameter ensemble comprised of parameter values read from the “parameter data” section of the PEST control file. The corresponding observation ensemble is comprised of measurements read from the “observation data” section of the PEST control file. |
| ies_restart_observation_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing model outputs calculated using a parameter ensemble. If it reads this file, PESTPP-IES does not calculate these itself, proceeding to upgrade calculations instead. |
| ies_restart_parameter_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing a parameter ensemble that corresponds to the ies_restart_observation_ensemble(). This option requires that the ies_restart_observation_ensemble() control variable also be supplied. This ensemble is only used in the calculation of the regularization component of the objective function for a restarted PESTPP-IES analysis. |
| ies_enforce_bounds(true) | Boolean | If set to true PESTPP-IES will not transgress bounds supplied in the PEST control file when generating or accepting parameter realizations, and when adjusting these realizations. |
| ies_initial_lambda() | real | The initial Marquardt lambda. The default value is \(10^{\text{floor}\left( \log_{10}\frac{\mu_{Փ}}{2n} \right)}\text{.\ \ }\)If supplied as a negative value, then the abs(ies_initial_lambda) is used as multiplier of the default initial-phi-based value. |
| ies_lambda_mults(0.1,1.0,10.0) | comma-separated reals | Factors by which to multiply the best lambda from the previous iteration to yield values for testing parameter upgrades during the current iteration. |
| lambda_scale_fac(0.75,1.0,1.1) | comma-separated reals | Line search factors along parameter upgrade directions computed using different Marquardt lambdas. |
| ies_subset_size(4) | integer | Number of realizations used in testing and evaluation of different Marquardt lambdas. If supplied as a negative value, then abs(ies_subset_size) is treated as a percentage of the current ensemble size – this allows the subset size to fluctuate with the size of the ensemble |
| ies_use_approx(true) | Boolean | Use complex or simple formula provided by Chen and Oliver (2013) for calculation of parameter upgrades. The more complex formula includes a function which constrains parameter realizations to respect prior means and probabilities. |
| ies_reg_factor(0.0) | real | Regularization objective function as a fraction of measurement objective function when constraining parameter realizations to respect initial values. |
| ies_bad_phi(1.0E300) | real | If the objective function calculated as an outcome of a model run is greater than this value, the model run is deemed to have failed. |
| ies_bad_phi_sigma(1.0E300) | real | If the objective function calculated for a given realization is greater than the current mean objective function of the ensemble plus the objective function standard deviation of the ensemble times ies_bad_phi_sigma(), that realization is treated as failed. If negative, its absolute value is treated as the upper quantile for identifying “bad” realizations. |
| ies_use_prior_scaling(false) | Boolean | Use a scaling factor based on the prior parameter distribution when evaluating parameter-to-model-output covariance used in calculation of the randomized Jacobian matrix. |
| ies_use_empirical_prior(false) | Boolean | Use an empirical, diagonal parameter covariance matrix for certain calculations. This matrix is contained in a file whose name is provided with the ies_parameter_ensemble() keyword. |
| Ies_save_lambda_ensembles(false) | Boolean | Save a set of CSV or JCB files that record parameter realizations used when testing different Marquardt lambdas. |
| ies_verbose_level(1) | 0, 1 or 2 | The level of diagnostic output provided by PESTPP-IES. If set to 2, all intermediate matrices are saved to ASCII files. This can require a considerable amount of storage. |
| ies_accept_phi_fac(1.05) | real > 1.0 | The factor applied to the previous best mean objective function to determine if the current mean objective function is acceptable. |
| ies_lambda_dec_fac(0.75) | real < 1.0 | The factor by which to decrease the value of the Marquardt lambda during the next IES iteration if the current iteration of the ensemble smoother process was successful in lowering the mean objective function. |
| ies_lambda_inc_fac(10.0) | real > 1.0 | The factor by which to increase the current value of the Marquardt lambda for further lambda testing if the current lambda testing cycle was unsuccessful. |
| ies_subset_how(random) | “first”,”last”, ”random”, ”phi_based | How to select the subset of realizations for objective function evaluation during upgrade testing. Default is “random”. |
| ies_num_threads(-1) | integer > 1 | The number of threads to use during the localized upgrade solution process, the automatic adaptive localization process. If the localizer contains many (>10K) rows, then multithreading can substantially speed up the upgrade calculation process. ies_num_threads() should not be greater than the number of physical cores on the host machine. |
| ies_localizer() | text | The name of a matrix to use for localization. The extension of the file is used to determine the type: .mat is an ASCII matrix file, .jcb/.jco signifies use of (enhanced) Jacobian matrix format (a binary format), while .csv signifies a comma-delimited file. Note that adjustable parameters not listed in localization matrix columns are implicitly treated as “fixed” while non-zero weighted observations not listed in rows of this matrix are implicitly treated as zero-weighted. |
| ies_group_draws(true) | Boolean | A flag to draw from the (multivariate) Gaussian prior by parameter/observation groups. This is usually a good idea since groups of parameters/observations are likely to have prior correlation. |
| ies_save_binary(false) | Boolean | A flag to save parameter and observation ensembles in binary (i.e., JCB) format instead of CSV format. |
| ies_csv_by_reals(true) | Boolean | A flag to save parameter and observation ensemble CSV files by realization instead of by variable name. If true, each row of the CSV file is a realization. If false, each column of the CSV file is a realization. |
| ies_autoadaloc(false) | Boolean | Flag to activate automatic adaptive localization. |
| ies_autoadaloc_sigma_dist(1.0) | Real | Real number representing the factor by which a correlation coefficient must exceed the standard deviation of background correlation coefficients to be considered significant. Default is 1.0 |
| tie_by_group(false) | Boolean | Flag to tie all adjustable parameters together within each parameter group. Initial parameter ratios are maintained as parameters are adjusted. Parameters that are designated as already tied, or that have parameters tied to them, are not affected. |
| ies_enforce_chglim(false) | Boolean | Flag to enforce parameter change limits (via FACPARMAX and RELPARMAX) in a way similar to PEST and PESTPP-GLM (by scaling the entire realization). Default is false. |
| ies_center_on() | String | A realization name that should be used for the ensemble center in calculating the approximate Jacobian matrix. The realization name must be in both the parameter and observation ensembles. If not passed, the mean vector is used as the center. The value “_MEDIAN_” can also be used, which instructs PESTPP-IES to use the median vector for calculating anomalies. |
| enforce_tied_bounds(false) | Boolean | Flag to enforce parameter bounds on any tied parameters. Depending on the ration between the tied and free parameters, this option can greatly limit parameter changes. |
| ies_no_noise(false) | Boolean | Flag to not generate and use realizations of measurement noise. Default is False (that is, to use measurement noise). |
| ies_drop_conflicts(false) | Boolean | Flag to remove non-zero weighted observations that are in a prior-data conflict state from the upgrade calculations. Default is False. |
| ies_pdc_sigma_distance() | Real > 0.0 | The number of standard deviations from the mean used in checking for prior-data conflict. |
| ies_save_rescov(False) | Boolean | Flag to save the iteration-level residual covariance matrix. If ies_save_binary is True, then a binary format file is written, otherwise an ASCII format (.cov) file is written. The file name is case.N.res.cov/.jcb. Note that this functionality does not scale beyond about 20,000 non-zero-weighted observations |
| obscov() | text | The name of a file containing the observation noise covariance matrix. This can be a parameter uncertainty file (extension .unc), a covariance matrix file (extension .cov) or a binary JCO or JCB file (extension .jco or .jcb). Please see the section on this matrix above to understand the implications of using this matrix |
| rand_seed(358183147) | unsigned integer | Seed for the random number generator. |
| Ies_use_mda(false) | Boolean | Flag to use the (optionally iterative) Kalman update equation – the number of data assimilation iterations is controlled by NOPTMAX; NOPTMAX = 1 and ies_use_mda(true) results in the standard ensemble smoother Kalman update. If False, the GLM iterative ensemble smoother equation is used. Default is False |
| Ies_mda_init_fac(10.0) | double | The initial MDA covariance inflation factor. Only used if ies_use_mda is true. Default is 10.0 |
| Ies_mda_decl_fac(0.5) | double | The final MDA covariance inflation factor. Only used in ies_use_mda is true. Default is 0.5 |
| Ies_localization_type(local) | text | Can be either “local” for local analysis or “covariance” for covariance-only localization. Default is “local” |
| Ies_upgrades_in_memory(true) | Boolean | Flag to hold parameter upgrade ensembles in memory during testing. If False, parameter ensembles are saved to disk during testing and the best-phi ensemble is loaded from disk after testing – this can reduce memory pressure for very high dimensional problems. Default is True but is only activated if number of parameters > 100K. |
| Ies_ordered_binary(true) | Boolean | Flag to write control-file-ordered binary ensemble files. Only used if save_binary is true. If false, hash-ordered binary files are written – for very high dimensional problems, writing unordered binary can save lots of time. If not passed and number of parameters > 100K, then ies_ordered_binary is set to false. |
| ensemble_output_precision(6) | int | Number of significant digits to use in ASCII format ensemble files. Default is 6 |
| ies_multimodal_alpha(1.0) | double | The fraction of the total ensemble size to use as the local neighborhood realizations in the multimodal solution process. Must be greater than zero and less than 1. Values of 0.1 to 0.25 seem to work well. Default is 1.0 (disable multi-modal solution process) |
| ies_weight_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing user-supplied weight vectors for each realization. If this keyword is omitted, PESTPP-IES uses the weight vector in the control file for all realizations. Only used with ies_multimodal_alpha |
| Ies_phi_factor_file | text | A two-column ASCII file that contains observation group “tags” and phi factors. Used to internally adjust weights to implement a balanced objective function using the mean residuals from the initial ensemble. |
| Ies_phi_factors_by_real | Bool | A flag to use internal weight balancing for each realization. This option should be used in conjunction with the multi-modal solution process. |
| Variable | Type | Role |
|---|---|---|
| ies_num_reals(50) | integer | The number of realizations to draw in order to form parameter and observation ensembles. |
| parcov() | text | The name of a file containing the prior parameter covariance matrix. This can be a parameter uncertainty file (extension .unc), a covariance matrix file (extension .cov) or a binary JCO or JCB file (extension .jco or .jcb). |
| par_sigma_range(4.0) | real | The difference between a parameter’s upper and lower bounds expressed as standard deviations. |
| ies_parameter_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing user-supplied parameter realizations comprising the initial (prior) parameter ensemble. If this keyword is omitted, PESTPP-IES generates the initial parameter ensemble itself. |
| ies_observation_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing user-supplied observation plus noise realizations comprising the observation plus noise ensemble. If this keyword is omitted, PESTPP-IES generates the observation plus noise ensemble itself. |
| ies_add_base(true) | Boolean | If set to true, instructs PESTPP-IES to include a “realization” in the initial parameter ensemble comprised of parameter values read from the “parameter data” section of the PEST control file. The corresponding observation ensemble is comprised of measurements read from the “observation data” section of the PEST control file. |
| ies_restart_observation_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing model outputs calculated using a parameter ensemble. If it reads this file, PESTPP-IES does not calculate these itself, proceeding to upgrade calculations instead. |
| ies_restart_parameter_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing a parameter ensemble that corresponds to the ies_restart_observation_ensemble(). This option requires that the ies_restart_observation_ensemble() control variable also be supplied. This ensemble is only used in the calculation of the regularization component of the objective function for a restarted PESTPP-IES analysis. |
| ies_enforce_bounds(true) | Boolean | If set to true PESTPP-IES will not transgress bounds supplied in the PEST control file when generating or accepting parameter realizations, and when adjusting these realizations. |
| ies_initial_lambda() | real | The initial Marquardt lambda. The default value is \(10^{\text{floor}\left( \log_{10}\frac{\mu_{Փ}}{2n} \right)}\text{.\ \ }\)If supplied as a negative value, then the abs(ies_initial_lambda) is used as multiplier of the default initial-phi-based value. |
| ies_lambda_mults(0.1,1.0,10.0) | comma-separated reals | Factors by which to multiply the best lambda from the previous iteration to yield values for testing parameter upgrades during the current iteration. |
| lambda_scale_fac(0.75,1.0,1.1) | comma-separated reals | Line search factors along parameter upgrade directions computed using different Marquardt lambdas. |
| ies_subset_size(4) | integer | Number of realizations used in testing and evaluation of different Marquardt lambdas. If supplied as a negative value, then abs(ies_subset_size) is treated as a percentage of the current ensemble size – this allows the subset size to fluctuate with the size of the ensemble |
| ies_use_approx(true) | Boolean | Use complex or simple formula provided by Chen and Oliver (2013) for calculation of parameter upgrades. The more complex formula includes a function which constrains parameter realizations to respect prior means and probabilities. |
| ies_reg_factor(0.0) | real | Regularization objective function as a fraction of measurement objective function when constraining parameter realizations to respect initial values. |
| ies_bad_phi(1.0E300) | real | If the objective function calculated as an outcome of a model run is greater than this value, the model run is deemed to have failed. |
| ies_bad_phi_sigma(1.0E300) | real | If the objective function calculated for a given realization is greater than the current mean objective function of the ensemble plus the objective function standard deviation of the ensemble times ies_bad_phi_sigma(), that realization is treated as failed. If negative, its absolute value is treated as the upper quantile for identifying “bad” realizations. |
| ies_use_prior_scaling(false) | Boolean | Use a scaling factor based on the prior parameter distribution when evaluating parameter-to-model-output covariance used in calculation of the randomized Jacobian matrix. |
| ies_use_empirical_prior(false) | Boolean | Use an empirical, diagonal parameter covariance matrix for certain calculations. This matrix is contained in a file whose name is provided with the ies_parameter_ensemble() keyword. |
| Ies_save_lambda_ensembles(false) | Boolean | Save a set of CSV or JCB files that record parameter realizations used when testing different Marquardt lambdas. |
| ies_verbose_level(1) | 0, 1 or 2 | The level of diagnostic output provided by PESTPP-IES. If set to 2, all intermediate matrices are saved to ASCII files. This can require a considerable amount of storage. |
| ies_accept_phi_fac(1.05) | real > 1.0 | The factor applied to the previous best mean objective function to determine if the current mean objective function is acceptable. |
| ies_lambda_dec_fac(0.75) | real < 1.0 | The factor by which to decrease the value of the Marquardt lambda during the next IES iteration if the current iteration of the ensemble smoother process was successful in lowering the mean objective function. |
| ies_lambda_inc_fac(10.0) | real > 1.0 | The factor by which to increase the current value of the Marquardt lambda for further lambda testing if the current lambda testing cycle was unsuccessful. |
| ies_subset_how(random) | “first”,”last”, ”random”, ”phi_based | How to select the subset of realizations for objective function evaluation during upgrade testing. Default is “random”. |
| ies_num_threads(-1) | integer > 1 | The number of threads to use during the localized upgrade solution process, the automatic adaptive localization process. If the localizer contains many (>10K) rows, then multithreading can substantially speed up the upgrade calculation process. ies_num_threads() should not be greater than the number of physical cores on the host machine. |
| ies_localizer() | text | The name of a matrix to use for localization. The extension of the file is used to determine the type: .mat is an ASCII matrix file, .jcb/.jco signifies use of (enhanced) Jacobian matrix format (a binary format), while .csv signifies a comma-delimited file. Note that adjustable parameters not listed in localization matrix columns are implicitly treated as “fixed” while non-zero weighted observations not listed in rows of this matrix are implicitly treated as zero-weighted. |
| ies_group_draws(true) | Boolean | A flag to draw from the (multivariate) Gaussian prior by parameter/observation groups. This is usually a good idea since groups of parameters/observations are likely to have prior correlation. |
| ies_save_binary(false) | Boolean | A flag to save parameter and observation ensembles in binary (i.e., JCB) format instead of CSV format. |
| ies_csv_by_reals(true) | Boolean | A flag to save parameter and observation ensemble CSV files by realization instead of by variable name. If true, each row of the CSV file is a realization. If false, each column of the CSV file is a realization. |
| ies_autoadaloc(false) | Boolean | Flag to activate automatic adaptive localization. |
| ies_autoadaloc_sigma_dist(1.0) | Real | Real number representing the factor by which a correlation coefficient must exceed the standard deviation of background correlation coefficients to be considered significant. Default is 1.0 |
| tie_by_group(false) | Boolean | Flag to tie all adjustable parameters together within each parameter group. Initial parameter ratios are maintained as parameters are adjusted. Parameters that are designated as already tied, or that have parameters tied to them, are not affected. |
| ies_enforce_chglim(false) | Boolean | Flag to enforce parameter change limits (via FACPARMAX and RELPARMAX) in a way similar to PEST and PESTPP-GLM (by scaling the entire realization). Default is false. |
| ies_center_on() | String | A realization name that should be used for the ensemble center in calculating the approximate Jacobian matrix. The realization name must be in both the parameter and observation ensembles. If not passed, the mean vector is used as the center. The value “_MEDIAN_” can also be used, which instructs PESTPP-IES to use the median vector for calculating anomalies. |
| enforce_tied_bounds(false) | Boolean | Flag to enforce parameter bounds on any tied parameters. Depending on the ration between the tied and free parameters, this option can greatly limit parameter changes. |
| ies_no_noise(false) | Boolean | Flag to not generate and use realizations of measurement noise. Default is False (that is, to use measurement noise). |
| ies_drop_conflicts(false) | Boolean | Flag to remove non-zero weighted observations that are in a prior-data conflict state from the upgrade calculations. Default is False. |
| ies_pdc_sigma_distance() | Real > 0.0 | The number of standard deviations from the mean used in checking for prior-data conflict. |
| ies_save_rescov(False) | Boolean | Flag to save the iteration-level residual covariance matrix. If ies_save_binary is True, then a binary format file is written, otherwise an ASCII format (.cov) file is written. The file name is case.N.res.cov/.jcb. Note that this functionality does not scale beyond about 20,000 non-zero-weighted observations |
| obscov() | text | The name of a file containing the observation noise covariance matrix. This can be a parameter uncertainty file (extension .unc), a covariance matrix file (extension .cov) or a binary JCO or JCB file (extension .jco or .jcb). Please see the section on this matrix above to understand the implications of using this matrix |
| rand_seed(358183147) | unsigned integer | Seed for the random number generator. |
| Ies_use_mda(false) | Boolean | Flag to use the (optionally iterative) Kalman update equation – the number of data assimilation iterations is controlled by NOPTMAX; NOPTMAX = 1 and ies_use_mda(true) results in the standard ensemble smoother Kalman update. If False, the GLM iterative ensemble smoother equation is used. Default is False |
| Ies_mda_init_fac(10.0) | double | The initial MDA covariance inflation factor. Only used if ies_use_mda is true. Default is 10.0 |
| Ies_mda_decl_fac(0.5) | double | The final MDA covariance inflation factor. Only used in ies_use_mda is true. Default is 0.5 |
| Ies_upgrades_in_memory(true) | Boolean | Flag to hold parameter upgrade ensembles in memory during testing. If False, parameter ensembles are saved to disk during testing and the best-phi ensemble is loaded from disk after testing – this can reduce memory pressure for very high dimensional problems. Default is True but is only activated if number of parameters > 100K. |
| Ies_ordered_binary(true) | Boolean | Flag to write control-file-ordered binary ensemble files. Only used if save_binary is true. If false, hash-ordered binary files are written – for very high dimensional problems, writing unordered binary can save lots of time. If not passed and number of parameters > 100K, then ies_ordered_binary is set to false. |
| ensemble_output_precision(6) | int | Number of significant digits to use in ASCII format ensemble files. Default is 6 |
| ies_multimodal_alpha(1.0) | double | The fraction of the total ensemble size to use as the local neighborhood realizations in the multimodal solution process. Must be greater than zero and less than 1. Values of 0.1 to 0.25 seem to work well. Default is 1.0 (disable multi-modal solution process) |
| ies_weight_ensemble() | text | The name of a CSV or JCO/JCB file (recognized by its extension) containing user-supplied weight vectors for each realization. If this keyword is omitted, PESTPP-IES uses the weight vector in the control file for all realizations. Only used with ies_multimodal_alpha |
| Ies_phi_factor_file | text | A two-column ASCII file that contains observation group “tags” and phi factors. Used to internally adjust weights to implement a balanced objective function using the mean residuals from the initial ensemble. |
| Ies_phi_factors_by_real | Bool | A flag to use internal weight balancing for each realization. This option should be used in conjunction with the multi-modal solution process. |