changed wording and default for multi_test_correction flag

docs/glossary.rst

@@ -83,7 +83,7 @@ You can find links to our detailed documentations for
 
 Subgroup analysis
 ------------------------------------
-Subgroup analysis in ExaAn will select subgroup (which is a segment of data) based on the input argument, and then perform a regular delta analysis per subgroup as described before.
+Subgroup analysis in ExpAn will select subgroup (which is a segment of data) based on the input argument, and then perform a regular delta analysis per subgroup as described before.
 
 That is to say, we don't compare between subgroups, but compare treatment with control within each subgroup.
 

docs/tutorial.rst

@@ -98,15 +98,15 @@ You may also find the description in our :ref:`API <modindex>` page.
 	* ``min_observations=20``: Minimum number of observations needed.
 	* ``nruns=10000``: Only used if assume normal is false.
 	* ``relative=False``: If relative==True, then the values will be returned as distances below and above the mean, respectively, rather than the absolute values.
-	* ``multi_test_correction=True``: Initiate multiple correction (Bonferroni correction is supported).
+	* ``multi_test_correction=False``: Initiate multiple correction (Bonferroni correction is supported).
 
 *group_sequential* is a frequentist approach for early stopping:
 
 	* ``spending_function='obrien_fleming'``: Currently we support only Obrient-Fleming alpha spending function for the frequentist early stopping decision.
 	* ``estimated_sample_size=None``: Sample size to be achieved towards the end of experiment. In other words, the actual size of data should be always smaller than estimated_sample_size.
 	* ``alpha=0.05``: Type-I error rate.
 	* ``cap=8``: Upper bound of the adapted z-score.
-	* ``multi_test_correction=True``: Initiate multiple correction (Bonferroni correction is supported).
+	* ``multi_test_correction=False``: Initiate multiple correction (Bonferroni correction is supported).
 
 *bayes_factor* is a Bayesian approach for delta analysis and early stopping:
 

expan/core/early_stopping.py

@@ -59,8 +59,8 @@ def group_sequential(x,
             the end of experiment
         alpha: type-I error rate
         cap: upper bound of the adapted z-score
-        multi_test_correction: multiple correction flag
-        num_tests: number of tests or reported kpis used for multiple correction (default: 1, no correction is done)
+        multi_test_correction (boolean): flag of whether the correction for multiple testing is needed
+        num_tests (integer): number of tests or reported kpis used for multiple correction
         
     Returns:
         EarlyStoppingStatistics object

expan/core/experiment.py

@@ -204,7 +204,7 @@ def sga(self, feature_name_to_bins, multi_test_correction=False):
         Args:
             feature_name_to_bins (dict): a dict of feature name (key) to list of Bin objects (value). 
                                       This dict defines how and on which column to perform the subgroup split.
-            multi_test_correction (boolean): True or False whether the correction for multiple testing is needed.
+            multi_test_correction (boolean): flag of whether the correction for multiple testing is needed.
         Returns:
             Analysis results per subgroup. 
         """
@@ -241,7 +241,7 @@ def sga_date(self, multi_test_correction=False):
         Perform subgroup analysis on date partitioning each day from start day till end date. Produces non-cumulative
         delta and CIs for each subgroup.
         Args:
-            multi_test_correction (boolean): True or False whether the correction for multiple testing is needed.
+            multi_test_correction (boolean): flag of whether the correction for multiple testing is needed.
         Returns:
             Analysis results per date
         """

expan/core/statistics.py

@@ -58,8 +58,8 @@ def delta(x, y, assume_normal=True, percentiles=[2.5, 97.5],
             the weighted mean and confidence intervals, which is equivalent
             to the overall metric. This weighted approach is only relevant
             for ratios.
-        multi_test_correction (boolean): correct the confidence intervals (multiple correction problem)
-        num_tests: number of tests or reported kpis.
+        multi_test_correction (boolean): flag of whether the correction for multiple testing is needed.
+        num_tests (integer): number of tests or reported kpis used for multiple correction.
 
     Returns:
         DeltaStatistics object
@@ -430,8 +430,8 @@ def normal_sample_difference(x, y, percentiles=[2.5, 97.5], relative=False, mult
             absolute values. In this case, the interval is mean-ret_val[0] to
             mean+ret_val[1]. This is more useful in many situations because it
             corresponds with the sem() and std() functions.
-        multi_test_correction (boolean): True or False whether the correction for multiple testing is needed.
-        num_tests: number of tests or reported kpis used for multiple correction (default: 1, no correction is done)
+        multi_test_correction (boolean): flag of whether the correction for multiple testing is needed.
+        num_tests (integer): number of tests or reported kpis used for multiple correction.
 
     Returns:
         dict: percentiles and corresponding values
@@ -477,8 +477,8 @@ def normal_difference(mean1, std1, n1, mean2, std2, n2, percentiles=[2.5, 97.5],
             absolute values. In	this case, the interval is mean-ret_val[0] to
             mean+ret_val[1]. This is more useful in many situations because it
             corresponds with the sem() and std() functions.
-        multi_test_correction (boolean): True or False whether the correction for multiple testing is needed.
-        num_tests: number of tests or reported kpis used for multiple correction (default: 1, no correction is done)
+        multi_test_correction (boolean): flag of whether the correction for multiple testing is needed.
+        num_tests (integer): number of tests or reported kpis used for multiple correction.
 
     Returns:
         dict: percentiles and corresponding values