Improve mp6 and bug fix on mt3dms (#1354)

* fix(mt3d/Mt3dBtn): stop text options on mt3dms

Mt3dBtn.write_file() wrote options regardless of version changed to only mt3dusgs

* fix(modpath.mp6, modpath.mp6sim, modpath.mp6bas): allow mp6 to run without modflow model and ensure user specified files and data preferred

implement reading layer,row,col,nper from dis file, remove unnecessary links to parent (modflow)
model so that the mp model can run without passing a modflow model. Ensure user specified ibound and laytyp is used unless they are None, in which case
read from parent modflow model
Create test t081_test.py to ensure behaviour.

* perf(mp6sim.StartingLocationsFile): Implement write particles with pandas

Implement option to write particles with pandas which increases write speeds more than 2x. User can specifiy original behaviour and will not use pandas if it cannot be imported Include test t082_test.py to ensure output is identical

* run pull_request_prepare.py (sorry)

* test(t081_test, t082_test): allow tests to complete without executables

allow tests to complete without executables (as per instructions from pull request review)

* style(mp6sim): reformat optional dependency import to match standards

reformat optional dependency (pandas) import to match standards as per pull request review

* style(mp6sim): reformat with python pull_request_prepare.py

reformat with python pull_request_prepare.py, (sorry for the two commits)

* fix(./etc, examples/data/mf6/test003_gwfs_disv):

address pull request comments

* fix(t049_test):

do not specify files, instead read from modflow object

* Update flopy3_modpath6_example.ipynb

fix(flopy3_modpath6_example.ipynb):

do not specify files (head, dis,), instead read from modflow object.  Previous example only worked because the head files were not handled correctly.

* fix(mt3d/Mt3dBtn): stop text options on mt3dms

Mt3dBtn.write_file() wrote options regardless of version changed to only mt3dusgs

* fix(./etc)

add mfpymake to etc environments so autotests run when using defined environments

* fix(modpath.mp6, modpath.mp6sim, modpath.mp6bas): allow mp6 to run without modflow model and ensure user specified files and data preferred

implement reading layer,row,col,nper from dis file, remove unnecessary links to parent (modflow)
model so that the mp model can run without passing a modflow model. Ensure user specified ibound and laytyp is used unless they are None, in which case
read from parent modflow model
Create test t081_test.py to ensure behaviour.

* perf(mp6sim.StartingLocationsFile): Implement write particles with pandas

Implement option to write particles with pandas which increases write speeds more than 2x. User can specifiy original behaviour and will not use pandas if it cannot be imported Include test t082_test.py to ensure output is identical

* run pull_request_prepare.py (sorry)

* style(mp6sim): reformat optional dependency import to match standards

reformat optional dependency (pandas) import to match standards as per pull request review

* style(mp6sim): reformat with python pull_request_prepare.py

reformat with python pull_request_prepare.py, (sorry for the two commits)

* fix(./etc, examples/data/mf6/test003_gwfs_disv):

address pull request comments

* test: update tests and remote mfpymake from etc/environment.yml

* fix(test_plot.py):: test_pathline_plot_xc

do not specify files, instead read from modflow object.  Previous version of modpath6 did not read the files but instead pulled the data from the model. Now if you want to specify the file you must pass the full path as it no longer assumes file is in the Modflow objects model_ws

* docs: update flopy3_modpath6_example notebook

* test: update test naming, use pytest-cases for mp6 tests, run linter

* docs: update flopy3_modpath6_example notebook

* run models silently (don't show paths from dev machine)
* mention change in Modpath6 constructor arguments
* follow temp workspace conventions from other notebooks

* test: move mp6 and mp7 tests into appropriate files, define more fixtures with pytest-cases

* test: catch ImportError when checking pkg availability

Co-authored-by: w-bonelli <wesbonelli@gmail.com>

.github/workflows/commit.yml

@@ -1,5 +1,4 @@
 name: FloPy continuous integration
-
 on:
   push:
     branches:
@@ -11,7 +10,6 @@ on:
     branches:
       - master
       - develop
-
 jobs:
   build:
     name: Build
@@ -20,7 +18,6 @@ jobs:
       run:
         shell: bash
     timeout-minutes: 10
-
     steps:
       - name: Checkout repo
         uses: actions/checkout@v2.3.4
@@ -276,7 +273,7 @@ jobs:
         uses: actions/cache@v3
         with:
           path: ~/conda_pkgs_dir
-          key: ${{ runner.os }}-${{ matrix.python-version }}-${{ matrix.run-type }}-${{ hashFiles('etc/environment.yml') }}
+          key: ${{ runner.os }}-${{ matrix.python-version }}-miniconda-${{ hashFiles('etc/environment.yml') }}
 
       # Standard python fails on windows without GDAL installation
       # Using custom bash shell ("shell: bash -l {0}") with Miniconda

autotest/conftest.py

@@ -192,7 +192,7 @@ def has_pkg(pkg):
         # pkg_resources expects package name, importlib expects import name
         try:
             _has_pkg_cache[pkg] = bool(importlib.import_module(pkg))
-        except ModuleNotFoundError:
+        except (ImportError, ModuleNotFoundError):
             try:
                 _has_pkg_cache[pkg] = bool(pkg_resources.get_distribution(pkg))
             except pkg_resources.DistributionNotFound:

autotest/test_export.py

@@ -13,7 +13,7 @@
     has_pkg,
     requires_exe,
     requires_pkg,
-    requires_spatial_reference,
+    requires_spatial_reference, excludes_platform,
 )
 from flaky import flaky
 
@@ -276,6 +276,7 @@ def test_export_shapefile_polygon_closed(tmpdir):
     shp.close()
 
 
+@excludes_platform("Windows")
 @requires_pkg("rasterio", "shapefile", "scipy")
 def test_export_array(tmpdir, example_data_path):
     import rasterio

autotest/test_modflow.py

@@ -10,9 +10,7 @@
     excludes_platform,
     get_example_data_path,
     requires_exe,
-    requires_pkg,
 )
-from matplotlib import pyplot as plt
 
 from flopy.discretization import StructuredGrid
 from flopy.mf6 import MFSimulation
@@ -31,9 +29,8 @@
     ModflowWel,
 )
 from flopy.mt3d import Mt3dBtn, Mt3dms
-from flopy.plot import PlotMapView
 from flopy.seawat import Seawat
-from flopy.utils import EndpointFile, PathlineFile, TimeseriesFile, Util2d
+from flopy.utils import Util2d
 
 
 @pytest.fixture
@@ -944,140 +941,6 @@ def test_rchload(tmpdir):
     assert np.allclose(a1, a2)
 
 
-@requires_pkg("pandas")
-def test_mp5_load(tmpdir, example_data_path):
-    # load the base freyberg model
-    freyberg_ws = example_data_path / "freyberg"
-    # load the modflow files for model map
-    m = Modflow.load(
-        "freyberg.nam",
-        model_ws=str(freyberg_ws),
-        check=False,
-        verbose=True,
-        forgive=False,
-    )
-
-    # load the pathline data
-    pthobj = PathlineFile(str(example_data_path / "mp5" / "m.ptl"))
-
-    # load endpoint data
-    fpth = str(example_data_path / "mp5" / "m.ept")
-    endobj = EndpointFile(fpth, verbose=True)
-
-    # determine version
-    ver = pthobj.version
-    assert ver == 5, f"{fpth} is not a MODPATH version 5 pathline file"
-
-    # read all of the pathline and endpoint data
-    plines = pthobj.get_alldata()
-    epts = endobj.get_alldata()
-
-    # determine the number of particles in the pathline file
-    nptl = pthobj.nid.shape[0]
-    assert nptl == 64, "number of MODPATH 5 particles does not equal 64"
-
-    hsv = plt.get_cmap("hsv")
-    colors = hsv(np.linspace(0, 1.0, nptl))
-
-    # plot the pathlines one pathline at a time
-    mm = PlotMapView(model=m)
-    for n in pthobj.nid:
-        p = pthobj.get_data(partid=n)
-        e = endobj.get_data(partid=n)
-        try:
-            mm.plot_pathline(p, colors=colors[n], layer="all")
-        except:
-            assert False, f'could not plot pathline {n + 1} with layer="all"'
-        try:
-            mm.plot_endpoint(e)
-        except:
-            assert False, f'could not plot endpoint {n + 1} with layer="all"'
-
-    # plot the grid and ibound array
-    mm.plot_grid(lw=0.5)
-    mm.plot_ibound()
-
-    fpth = os.path.join(str(tmpdir), "mp5.pathline.png")
-    plt.savefig(fpth, dpi=300)
-    plt.close()
-
-
-@requires_pkg("pandas")
-def test_mp5_timeseries_load(example_data_path):
-    pth = str(example_data_path / "mp5")
-    files = [
-        os.path.join(pth, name)
-        for name in sorted(os.listdir(pth))
-        if ".timeseries" in name
-    ]
-    for file in files:
-        print(file)
-        eval_timeseries(file)
-
-
-@requires_pkg("pandas")
-def test_mp6_timeseries_load(example_data_path):
-    pth = str(example_data_path / "mp5")
-    files = [
-        os.path.join(pth, name)
-        for name in sorted(os.listdir(pth))
-        if ".timeseries" in name
-    ]
-    for file in files:
-        print(file)
-        eval_timeseries(file)
-
-
-def eval_timeseries(file):
-    ts = TimeseriesFile(file)
-    assert isinstance(ts, TimeseriesFile), (
-        f"{os.path.basename(file)} " "is not an instance of TimeseriesFile"
-    )
-
-    # get the all of the data
-    tsd = ts.get_alldata()
-    assert (
-        len(tsd) > 0
-    ), f"could not load data using get_alldata() from {os.path.basename(file)}."
-
-    # get the data for the last particleid
-    partid = ts.get_maxid()
-    assert partid is not None, (
-        "could not get maximum particleid using get_maxid() from "
-        f"{os.path.basename(file)}."
-    )
-
-    tsd = ts.get_data(partid=partid)
-    assert tsd.shape[0] > 0, (
-        f"could not load data for particleid {partid} using get_data() from "
-        f"{os.path.basename(file)}. Maximum partid = {ts.get_maxid()}."
-    )
-
-    timemax = ts.get_maxtime() / 2.0
-    assert timemax is not None, (
-        "could not get maximum time using get_maxtime() from "
-        f"{os.path.basename(file)}."
-    )
-
-    tsd = ts.get_alldata(totim=timemax)
-    assert len(tsd) > 0, (
-        f"could not load data for totim>={timemax} using get_alldata() from "
-        f"{os.path.basename(file)}. Maximum totim = {ts.get_maxtime()}."
-    )
-
-    timemax = ts.get_maxtime()
-    assert timemax is not None, (
-        "could not get maximum time using get_maxtime() from "
-        f"{os.path.basename(file)}."
-    )
-
-    tsd = ts.get_alldata(totim=timemax, ge=False)
-    assert len(tsd) > 0, (
-        f"could not load data for totim<={timemax} using get_alldata() from "
-        f"{os.path.basename(file)}. Maximum totim = {ts.get_maxtime()}."
-    )
-
-
 def test_default_oc_stress_period_data(tmpdir):
     m = Modflow(model_ws=str(tmpdir), verbose=True)
     dis = ModflowDis(m, nper=10, perlen=10.0, nstp=5)

autotest/test_mp5.py

@@ -0,0 +1,81 @@
+import os
+
+import numpy as np
+from autotest.conftest import requires_pkg
+from autotest.test_mp6 import eval_timeseries
+from matplotlib import pyplot as plt
+
+from flopy.modflow import Modflow
+from flopy.plot import PlotMapView
+from flopy.utils import EndpointFile, PathlineFile
+
+
+@requires_pkg("pandas")
+def test_mp5_load(tmpdir, example_data_path):
+    # load the base freyberg model
+    freyberg_ws = example_data_path / "freyberg"
+    # load the modflow files for model map
+    m = Modflow.load(
+        "freyberg.nam",
+        model_ws=str(freyberg_ws),
+        check=False,
+        verbose=True,
+        forgive=False,
+    )
+
+    # load the pathline data
+    pthobj = PathlineFile(str(example_data_path / "mp5" / "m.ptl"))
+
+    # load endpoint data
+    fpth = str(example_data_path / "mp5" / "m.ept")
+    endobj = EndpointFile(fpth, verbose=True)
+
+    # determine version
+    ver = pthobj.version
+    assert ver == 5, f"{fpth} is not a MODPATH version 5 pathline file"
+
+    # read all of the pathline and endpoint data
+    plines = pthobj.get_alldata()
+    epts = endobj.get_alldata()
+
+    # determine the number of particles in the pathline file
+    nptl = pthobj.nid.shape[0]
+    assert nptl == 64, "number of MODPATH 5 particles does not equal 64"
+
+    hsv = plt.get_cmap("hsv")
+    colors = hsv(np.linspace(0, 1.0, nptl))
+
+    # plot the pathlines one pathline at a time
+    mm = PlotMapView(model=m)
+    for n in pthobj.nid:
+        p = pthobj.get_data(partid=n)
+        e = endobj.get_data(partid=n)
+        try:
+            mm.plot_pathline(p, colors=colors[n], layer="all")
+        except:
+            assert False, f'could not plot pathline {n + 1} with layer="all"'
+        try:
+            mm.plot_endpoint(e)
+        except:
+            assert False, f'could not plot endpoint {n + 1} with layer="all"'
+
+    # plot the grid and ibound array
+    mm.plot_grid(lw=0.5)
+    mm.plot_ibound()
+
+    fpth = os.path.join(str(tmpdir), "mp5.pathline.png")
+    plt.savefig(fpth, dpi=300)
+    plt.close()
+
+
+@requires_pkg("pandas")
+def test_mp5_timeseries_load(example_data_path):
+    pth = str(example_data_path / "mp5")
+    files = [
+        os.path.join(pth, name)
+        for name in sorted(os.listdir(pth))
+        if ".timeseries" in name
+    ]
+    for file in files:
+        print(file)
+        eval_timeseries(file)