WIP 0.8.0 (#106)

* Store fits and residuals in Lm2 models. Also compute model r2 metrics * Remove manual rhome set during dev * lrt for lmer objects Implement loglik ratio test analogous to the one implemented in lme4 * Fix pandas deprecated function * Add BIC and deviance to lrt Make output closer to lme4 * Match behaviour in R Make sure npar are correct and other values are in the same order * Format code with black * Get correct AIC AICTab in summary of an lmer fitted by REML gives the deviance, not the AIC. * Add tests for lrt For the moment, we use a static dump of the R output, as rpy2 seems to clash with pytest * Add confint to test variances * Add tests for confint * Guard against using lrt for non Lmer models * Temp fix for rpy2 3.5.1, but not until they fix #873 recursion error * Bump rpy2 and pandas requirements * Update dep versions code changes to address rpy2 deprecation warnings * try fixing GA ci * try fixing GA ci * rename meta.yaml and bump numpy version * try fixing GA ci * try fixing GA ci * remove deprecated future imports * refactor code to pull out R<->Py conversion into a separate module and remove global conversion activators * fix bug in saving confints * fixes #88, #113 * add sklearn dependency * initial implementation code * use pytest fixtures for data loading * complete working (basic) implementation of Logistic Regression * logistic Lm estimate test against glm() in R * add rpy2 back to requirements * reorg workflow files to run tests only on each push/pr * whoops remove dev branch ref * Update Tests.yml * try fixing GA ci * try fixing GA ci * try fixing GA ci * install rpy2 from conda-forge instead of pip to see if rpy2 linking works on macos * install rpy2 from conda-forge instead of pip to see if rpy2 linking works on macos * try old optimizer for inverse_gaussian model * comment out rfx only models that crash only on GA * GA try continuing on expected failure * fix up conda build * fix ci * Update Tests.yml * Update Build.yml * fix ci * update meta.yaml for working local build * Update Tests.yml * store fits in logistic Lm and add support for .predict * allow .predict to return probs or original scale vals. Fix bug in converting logits to probs in Lmer. Fixes #78" * add sphinx to dev reqs and update changelog * update gitignore * refactor Lm a bit Co-authored-by: Andrea Manica <am315@cam.ac.uk>
ejolly · Dec 9, 2022 · 9550ae0 · 9550ae0
1 parent 4d18676
commit 9550ae0
Show file tree

Hide file tree

Showing 22 changed files with 1,445 additions and 677 deletions.
diff --git a/.github/workflows/Tests.yml b/.github/workflows/Tests.yml
@@ -59,7 +59,8 @@ jobs:
           PY_VER: ${{ matrix.py_ver }}
         run: |
           conda config --set always_yes yes --set changeps1 no
-          conda create -n pkg_test -c conda-forge python=$PY_VER 'r-lmerTest' 'r-emmeans' rpy2
+          conda create -n pkg_test -c conda-forge python=$PY_VER 'r-lmerTest' 'r-emmeans' 'rpy2'
+
           conda activate pkg_test
           pip install -r requirements-dev.txt
           pip install .

diff --git a/.gitignore b/.gitignore
@@ -17,3 +17,9 @@ dev/
 .vscode/settings.json
 *.key
 pytest.ini
+env/*
+conda/build/*
+.pytest_cache
+.tox
+.venv
+environment.yml
diff --git a/README.md b/README.md
@@ -53,6 +53,29 @@ Check out the [documentation site](http://eshinjolly.com/pymer4/) for detailed t
 
 Installing via [Anaconda](https://www.anacnda.com/products/individual) is the preferred installation method. Follow the directions [here](https://eshinjolly.com/pymer4/installation.html).  
 
+## Local build
+
+```
+conda create -n build python=3.8 pip conda-build
+conda activate build
+conda-build ./conda --python=3.8 -c https://conda.anaconda.org/conda-forge --output-folder ./conda/build --output ./conda/build
+tarball=$(conda-build ./conda --python=3.8 --output | tail -1)
+conda convert -p linux-64 -p win-64 $tarball -o ./conda/build
+```
+
+## Local upload
+
+```
+# after
+# conda install anaconda-client
+# anaconda login
+
+# pre-release
+anaconda upload ./conda/build/**/pymer4*.tar.bz2 -l "pre-release"
+
+# main
+anaconda upload ./conda/build/**/pymer4*.tar.bz2 -l "main"
+```
 
 ## Contributing
 

diff --git a/conda/conda_build_config.yaml b/conda/conda_build_config.yaml
diff --git a/conda/meta.yaml b/conda/meta.yaml
@@ -10,7 +10,6 @@ source:
     path_url: ../ # so conda build will grab source to read version
 
 build:
-    # script: python setup.py install --single-version-externally-managed --record=record.txt
     script: pip install . -vv --no-deps
     # abandoned conda build GIT_BUILD_STRING env variable
     string: py{{environ.get("CONDA_PY", "XX")}}{{environ.get("GIT_ABBREV_COMMIT", "no_git_abbrev_commit") }}_{{ environ.get("PKG_BUILDNUM", "no_pkg_buildnum") }}
@@ -19,22 +18,37 @@ requirements:
 
   host:
     - python {{ python }}
+    # - r-base
+    # - r-lmerTest
+    # - r-base
     - pip
+    - numpy >=1.20
 
   run:
     - python {{ python }}
     - r-lmerTest
     - r-emmeans
-    {% for req in data.get('install_requires', []) %}
-      - {{ req }}
-    {% endfor %}
+    - pandas >=1.1.0
+    - numpy >=1.20
+    - rpy2 >=3.5.3
+    - seaborn
+    - matplotlib
+    - patsy
+    - joblib
+    - scipy
+    - deepdish
+    - scikit-learn
+    # {% for req in data.get('install_requires', []) %}
+    # - {{ req }}
+    # {% endfor %}
 
 test:
   imports:
    - pymer4
    - pymer4.models
   requires:
    - pytest
+   - black
 
 about:
   home: "https://eshinjolly.com/pymer4/"

diff --git a/docs/auto_examples/index.rst b/docs/auto_examples/index.rst
@@ -1,9 +1,5 @@
 :orphan:
 
-
-
-.. _sphx_glr_auto_examples:
-
 Usage Guide
 ===========
 
@@ -21,131 +17,122 @@ Usage examples:
 
 
 
+.. raw:: html
+
+    <div class="sphx-glr-thumbnails">
+
+
 .. raw:: html
 
     <div class="sphx-glr-thumbcontainer" tooltip="1. Basic Usage Guide">
 
 .. only:: html
 
- .. figure:: /auto_examples/images/thumb/sphx_glr_example_01_basic_usage_thumb.png
-     :alt: 1. Basic Usage Guide
+  .. image:: /auto_examples/images/thumb/sphx_glr_example_01_basic_usage_thumb.png
+    :alt: 1. Basic Usage Guide
 
-     :ref:`sphx_glr_auto_examples_example_01_basic_usage.py`
+  :ref:`sphx_glr_auto_examples_example_01_basic_usage.py`
 
 .. raw:: html
 
+      <div class="sphx-glr-thumbnail-title">1. Basic Usage Guide</div>
     </div>
 
 
-.. toctree::
-   :hidden:
-
-   /auto_examples/example_01_basic_usage
-
 .. raw:: html
 
     <div class="sphx-glr-thumbcontainer" tooltip="2. Categorical Predictors">
 
 .. only:: html
 
- .. figure:: /auto_examples/images/thumb/sphx_glr_example_02_categorical_thumb.png
-     :alt: 2. Categorical Predictors
+  .. image:: /auto_examples/images/thumb/sphx_glr_example_02_categorical_thumb.png
+    :alt: 2. Categorical Predictors
 
-     :ref:`sphx_glr_auto_examples_example_02_categorical.py`
+  :ref:`sphx_glr_auto_examples_example_02_categorical.py`
 
 .. raw:: html
 
+      <div class="sphx-glr-thumbnail-title">2. Categorical Predictors</div>
     </div>
 
 
-.. toctree::
-   :hidden:
-
-   /auto_examples/example_02_categorical
-
 .. raw:: html
 
     <div class="sphx-glr-thumbcontainer" tooltip="3. ANOVA tables and post-hoc comparisons">
 
 .. only:: html
 
- .. figure:: /auto_examples/images/thumb/sphx_glr_example_03_posthoc_thumb.png
-     :alt: 3. ANOVA tables and post-hoc comparisons
+  .. image:: /auto_examples/images/thumb/sphx_glr_example_03_posthoc_thumb.png
+    :alt: 3. ANOVA tables and post-hoc comparisons
 
-     :ref:`sphx_glr_auto_examples_example_03_posthoc.py`
+  :ref:`sphx_glr_auto_examples_example_03_posthoc.py`
 
 .. raw:: html
 
+      <div class="sphx-glr-thumbnail-title">3. ANOVA tables and post-hoc comparisons</div>
     </div>
 
 
-.. toctree::
-   :hidden:
-
-   /auto_examples/example_03_posthoc
-
 .. raw:: html
 
     <div class="sphx-glr-thumbcontainer" tooltip="pymer4 gives you a lot of control over what you want your data to look like by setting properti...">
 
 .. only:: html
 
- .. figure:: /auto_examples/images/thumb/sphx_glr_example_04_simulating_data_thumb.png
-     :alt: 4. Simulating Data
+  .. image:: /auto_examples/images/thumb/sphx_glr_example_04_simulating_data_thumb.png
+    :alt: 4. Simulating Data
 
-     :ref:`sphx_glr_auto_examples_example_04_simulating_data.py`
+  :ref:`sphx_glr_auto_examples_example_04_simulating_data.py`
 
 .. raw:: html
 
+      <div class="sphx-glr-thumbnail-title">4. Simulating Data</div>
     </div>
 
 
-.. toctree::
-   :hidden:
-
-   /auto_examples/example_04_simulating_data
-
 .. raw:: html
 
     <div class="sphx-glr-thumbcontainer" tooltip="5. Additional Statistics Functions">
 
 .. only:: html
 
- .. figure:: /auto_examples/images/thumb/sphx_glr_example_05_misc_stats_thumb.png
-     :alt: 5. Additional Statistics Functions
+  .. image:: /auto_examples/images/thumb/sphx_glr_example_05_misc_stats_thumb.png
+    :alt: 5. Additional Statistics Functions
 
-     :ref:`sphx_glr_auto_examples_example_05_misc_stats.py`
+  :ref:`sphx_glr_auto_examples_example_05_misc_stats.py`
 
 .. raw:: html
 
+      <div class="sphx-glr-thumbnail-title">5. Additional Statistics Functions</div>
     </div>
 
 
-.. toctree::
-   :hidden:
-
-   /auto_examples/example_05_misc_stats
 .. raw:: html
 
-    <div class="sphx-glr-clear"></div>
-
+    </div>
 
 
-.. only :: html
+.. toctree::
+   :hidden:
 
- .. container:: sphx-glr-footer
-    :class: sphx-glr-footer-gallery
+   /auto_examples/example_01_basic_usage
+   /auto_examples/example_02_categorical
+   /auto_examples/example_03_posthoc
+   /auto_examples/example_04_simulating_data
+   /auto_examples/example_05_misc_stats
 
 
-  .. container:: sphx-glr-download sphx-glr-download-python
+.. only:: html
 
-    :download:`Download all examples in Python source code: auto_examples_python.zip </auto_examples/auto_examples_python.zip>`
+  .. container:: sphx-glr-footer sphx-glr-footer-gallery
 
+    .. container:: sphx-glr-download sphx-glr-download-python
 
+      :download:`Download all examples in Python source code: auto_examples_python.zip </auto_examples/auto_examples_python.zip>`
 
-  .. container:: sphx-glr-download sphx-glr-download-jupyter
+    .. container:: sphx-glr-download sphx-glr-download-jupyter
 
-    :download:`Download all examples in Jupyter notebooks: auto_examples_jupyter.zip </auto_examples/auto_examples_jupyter.zip>`
+      :download:`Download all examples in Jupyter notebooks: auto_examples_jupyter.zip </auto_examples/auto_examples_jupyter.zip>`
 
 
 .. only:: html

diff --git a/docs/new.rst b/docs/new.rst
@@ -2,6 +2,35 @@ What's New
 ==========
 Historically :code:`pymer4` versioning was a bit all over the place but has settled down since 0.5.0. This page includes the most notable updates between versions but github is the best place to checkout more details and `releases <https://github.com/ejolly/pymer4/releases/>`_.
 
+0.8.0
+----
+- **NOTE:**
+    - there was no 0.7.9 release as there were enough major changes to warrant a new
+      minor release version
+    - this version unpins the maximum versions of :code:`rpy2` and :code:`pandas`
+    - if there are install issues with the :code:`conda` release accompanying this version you should be able to successfully install into a conda environment using pip with the following: :code:`conda install 'r-lmerTest' 'r-emmeans' rpy2 -c conda-forge` followed by :code:`pip install pymer4`
+- **Bug fixes:**
+    - fixed as issue where :code:`Lmer` with :code:`family='binomial'` was not
+      converting logits into probabilities correctly
+    - fixes `#79 <https://github.com/ejolly/pymer4/issues/79>`_
+    - fixes `#88 <https://github.com/ejolly/pymer4/issues/88>`_
+    - fixes `#113 <https://github.com/ejolly/pymer4/issues/113>`_
+    - fixes `#114 <https://github.com/ejolly/pymer4/issues/114>`_ 
+    - generally more robust conversion of R types to pandas 
+
+- **New features:**
+    - :code:`Lm` models now support :code:`family='binomial'` and uses the
+      :code:`LogisticRegression` class from scikit-learn with no regularization for
+      estimation. Estimates and errors have been verified against the :code:`glm`
+      implementation in R
+    - new :code:`lrt` function for estimating likelihood-ratio tests between
+      :code:`Lmer` models thanks to `@dramanica <https://github.com/dramanica>`_. This
+      replicates the functionality of :code:`anova()` in R for :code:`lmer` models. 
+    - new :code:`.confint()` method for :code:`Lmer` models thanks to `@dramanica
+      <https://github.com/dramanica>`_. This allows computing confidence intervals on 1
+      or more paramters of an already fit model including random effects which are not
+      computed by default when calling :code:`.fit()`
+
 0.7.8
 -----
 - Maintenance release that pins :code:`rpy2 >= 3.4.5,< 3.5.1` due to R -> Python dataframe conversion issue on recent :code:`rpy2` versions that causes a `recursion error <https://github.com/rpy2/rpy2/issues/866>`_. 

diff --git a/pymer4/__init__.py b/pymer4/__init__.py
@@ -1,28 +1,10 @@
-from __future__ import absolute_import
-
 __all__ = ["models", "utils", "simulate", "stats", "io", "__version__"]
 
 from .models import Lmer, Lm, Lm2
-from .simulate import easy_multivariate_normal, simulate_lm, simulate_lmm
-
-from .utils import (
-    get_resource_path,
-    isPSD,
-    nearestPSD,
-    upper,
-    R2con,
-    con2R,
-    result_to_table,
-)
-from .io import save_model, load_model
-from .stats import (
-    discrete_inverse_logit,
-    cohens_d,
-    perm_test,
-    tost_equivalence,
-    boot_func,
-    welch_dof,
-    vif,
-)
+from .simulate import *
 
+from .utils import *
+from .io import *
+from .stats import *
+from .bridge import *
 from .version import __version__