Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adopt isort #4174

Merged
merged 13 commits into from
Jun 18, 2021
Merged

Adopt isort #4174

merged 13 commits into from
Jun 18, 2021

Conversation

bjlittle
Copy link
Member

@bjlittle bjlittle commented Jun 3, 2021

🚀 Pull Request

Description

This PR adopts isort as a means to automate the process of managing and ordering our module imports.

This PR includes an associated isort check as part of the cirrus-ci linting task, and a pre-commit git hook for developers, both of which will keep us honest from this point onwards.

I've intentionally not included isort as a PyPI or conda dependency; let's not unnecessarily add to the dependency bloat, developers can install isort easily themselves - otherwise pre-commit is your new best friend.

In order to get this PR across the line, I've had to tackle some really pretty ancient baked-in technical debt with regards to import order. Over the years we appear to have skilfully crafted a dependency monster, and tip-toed deftly around some truly awful circular dependencies... needless to say, isort pretty much uncovered them all.

That said, thanks to pydeps, which is a true gem of a package, it was reasonably straight forward to find dependency cycles (pydeps --show-cycles) and break them, one by one.


Consult Iris pull request check list

@bjlittle
Copy link
Member Author

bjlittle commented Jun 3, 2021

The side-effect from taking a wholistic approach to refreshing the import ordering means that import iris is now quite a bit faster; I'm seeing a rough 6-7 times speed-up (only tested on an NFS mounted FS within a VDI).

This is simply because we're now only importing the minimum required packages; I'm sure we can refine this further as we move forwards e.g., adopt lazy imports 🤔

We've gone from an ~850ms "import iris" time with Iris "as-is"
> python -X importtime -c "import iris"
import time: self [us] | cumulative | imported package
import time:       250 |        250 |   _io
import time:        46 |         46 |   marshal
import time:       652 |        652 |   posix
import time:       645 |       1591 | _frozen_importlib_external
import time:       133 |        133 |   time
import time:       244 |        376 | zipimport
import time:        58 |         58 |     _codecs
import time:       528 |        585 |   codecs
import time:       382 |        382 |   encodings.aliases
import time:       696 |       1663 | encodings
import time:       214 |        214 | encodings.utf_8
import time:       198 |        198 | _signal
import time:       846 |        846 | encodings.latin_1
import time:        48 |         48 |     _abc
import time:       305 |        352 |   abc
import time:       358 |        709 | io
import time:        63 |         63 |       _stat
import time:       347 |        410 |     stat
import time:      1075 |       1075 |     _collections_abc
import time:       200 |        200 |       genericpath
import time:       293 |        492 |     posixpath
import time:       669 |       2643 |   os
import time:       202 |        202 |   _sitebuiltins
import time:        85 |         85 |     _locale
import time:       158 |        242 |   _bootlocale
import time:       267 |        267 |   types
import time:       267 |        267 |       warnings
import time:       190 |        457 |     importlib
import time:       156 |        156 |       importlib.machinery
import time:       502 |        658 |     importlib.abc
import time:        88 |         88 |           _operator
import time:       314 |        401 |         operator
import time:       147 |        147 |         keyword
import time:       315 |        315 |           _heapq
import time:       192 |        507 |         heapq
import time:       104 |        104 |         itertools
import time:       205 |        205 |         reprlib
import time:       716 |        716 |         _collections
import time:       854 |       2933 |       collections
import time:        59 |         59 |         _functools
import time:       592 |        650 |       functools
import time:       602 |       4184 |     contextlib
import time:       434 |       5730 |   importlib.util
import time:       499 |        499 |   sitecustomize
import time:        83 |         83 |   usercustomize
import time:      3828 |      13492 | site
import time:       770 |        770 |       enum
import time:       212 |        212 |         _sre
import time:       340 |        340 |           sre_constants
import time:       413 |        753 |         sre_parse
import time:       333 |       1297 |       sre_compile
import time:       231 |        231 |       copyreg
import time:       641 |       2937 |     re
import time:       187 |        187 |     fnmatch
import time:       458 |       3581 |   glob
import time:       252 |        252 |     _weakrefset
import time:       666 |        917 |   threading
import time:       208 |        208 |       collections.abc
import time:      2049 |       2256 |     configparser
import time:       209 |        209 |             token
import time:      1014 |       1223 |           tokenize
import time:       212 |       1434 |         linecache
import time:       332 |       1766 |       traceback
import time:       533 |        533 |       weakref
import time:        49 |         49 |         _string
import time:       842 |        890 |       string
import time:        55 |         55 |       atexit
import time:      2468 |       5710 |     logging
import time:      2805 |      10770 |   iris.config
import time:       378 |        378 |           org
import time:        27 |        405 |         org.python
import time:        21 |        425 |       org.python.core
import time:       391 |        816 |     copy
import time:       241 |        241 |         xml
import time:       262 |        262 |         xml.dom.domreg
import time:      1059 |       1561 |       xml.dom
import time:       296 |        296 |       xml.dom.minicompat
import time:       261 |        261 |         xml.dom.NodeFilter
import time:       563 |        824 |       xml.dom.xmlbuilder
import time:      1337 |       4017 |     xml.dom.minidom
import time:       462 |        462 |     zlib
import time:       947 |        947 |             _ast
import time:       585 |       1532 |           ast
import time:      1780 |       1780 |               _struct
import time:       257 |       2037 |             struct
import time:       246 |        246 |             binascii
import time:       526 |       2807 |           base64
import time:       225 |        225 |                 _json
import time:       606 |        831 |               json.scanner
import time:       654 |       1484 |             json.decoder
import time:      1531 |       1531 |             json.encoder
import time:       303 |       3317 |           json
import time:       336 |        336 |             yaml.error
import time:       312 |        312 |             yaml.tokens
import time:       289 |        289 |             yaml.events
import time:       286 |        286 |             yaml.nodes
import time:      5629 |       5629 |               yaml.reader
import time:       567 |        567 |               yaml.scanner
import time:       286 |        286 |               yaml.parser
import time:       258 |        258 |               yaml.composer
import time:       299 |        299 |                   math
import time:       374 |        374 |                   _datetime
import time:       934 |       1607 |                 datetime
import time:      1224 |       2830 |               yaml.constructor
import time:      2280 |       2280 |               yaml.resolver
import time:       411 |      12259 |             yaml.loader
import time:       416 |        416 |               yaml.emitter
import time:       272 |        272 |               yaml.serializer
import time:       395 |        395 |               yaml.representer
import time:       320 |       1400 |             yaml.dumper
import time:        55 |         55 |                 yaml.yaml
import time:       423 |        478 |               yaml._yaml
import time:       377 |        854 |             yaml.cyaml
import time:       479 |      16211 |           yaml
import time:     10656 |      34521 |         dask.config
import time:       231 |        231 |               _bisect
import time:       185 |        416 |             bisect
import time:       182 |        182 |             _sha512
import time:       174 |        174 |             _random
import time:       515 |       1286 |           random
import time:       181 |        181 |                   _opcode
import time:       318 |        498 |                 opcode
import time:       600 |       1098 |               dis
import time:      1882 |       2979 |             inspect
import time:        82 |         82 |               errno
import time:       283 |        283 |                 _compression
import time:       235 |        235 |                 _bz2
import time:       386 |        903 |               bz2
import time:       367 |        367 |                 _lzma
import time:       298 |        665 |               lzma
import time:        47 |         47 |               pwd
import time:       181 |        181 |               grp
import time:       693 |       2569 |             shutil
import time:       508 |        508 |             tempfile
import time:      2107 |       2107 |               platform
import time:       890 |        890 |                 signal
import time:       110 |        110 |                 msvcrt
import time:       186 |        186 |                 _posixsubprocess
import time:       202 |        202 |                 select
import time:       689 |        689 |                 selectors
import time:       664 |       2737 |               subprocess
import time:       277 |        277 |               _uuid
import time:      6344 |      11465 |             uuid
import time:       818 |        818 |             numbers
import time:      3102 |       3102 |             typing
import time:       195 |        195 |               dask.utils_test
import time:       299 |        494 |             dask.core
import time:      1218 |      23150 |           dask.utils
import time:       294 |      24729 |         dask.datasets
import time:       305 |        305 |             concurrent
import time:       801 |        801 |             concurrent.futures._base
import time:       325 |       1430 |           concurrent.futures
import time:       266 |        266 |             _queue
import time:       334 |        599 |           queue
import time:       215 |        215 |           dask.callbacks
import time:       233 |        233 |           dask.order
import time:       364 |       2838 |         dask.local
import time:       762 |        762 |           dataclasses
import time:       147 |        147 |                   toolz.utils
import time:       414 |        560 |                 toolz.itertoolz
import time:      1304 |       1304 |                   textwrap
import time:       585 |        585 |                   toolz._signatures
import time:       450 |       2337 |                 toolz.functoolz
import time:       206 |        206 |                 toolz.dicttoolz
import time:       140 |        140 |                 toolz.recipes
import time:       198 |        198 |                     __future__
import time:     20059 |      20257 |                   toolz.curried.operator
import time:       263 |        263 |                   toolz.curried.exceptions
import time:       366 |      20886 |                 toolz.curried
import time:       245 |        245 |                   toolz.sandbox.core
import time:       147 |        147 |                   toolz.sandbox.parallel
import time:       220 |        611 |                 toolz.sandbox
import time:       231 |        231 |                 toolz._version
import time:      4020 |      28988 |               toolz
import time:        71 |         71 |                 cytoolz.collections
import time:        40 |         40 |                 cytoolz.heapq
import time:        34 |         34 |                 cytoolz.itertools
import time:        31 |         31 |                 cytoolz.operator
import time:        30 |         30 |                 cytoolz.random
import time:        34 |         34 |                   cytoolz.cytoolz
import time:        15 |         48 |                 cytoolz.cytoolz.utils
import time:        40 |         40 |                     cytoolz.os
import time:        25 |         64 |                   cytoolz.os.path
import time:        32 |         32 |                   cytoolz.cytoolz
import time:        28 |         28 |                     cytoolz.toolz
import time:        13 |         40 |                   cytoolz.toolz.utils
import time:       226 |        361 |                 cytoolz.utils
import time:      1490 |       2103 |               cytoolz.itertoolz
import time:       130 |        130 |                 backports_abc
import time:        39 |         39 |                 cytoolz.inspect
import time:        47 |         47 |                 cytoolz.sys
import time:        35 |         35 |                 cytoolz.functools
import time:        31 |         31 |                 cytoolz.importlib
import time:        30 |         30 |                 cytoolz.operator
import time:        29 |         29 |                 cytoolz.textwrap
import time:        29 |         29 |                 cytoolz.types
import time:        28 |         28 |                   cytoolz.cytoolz
import time:        14 |         42 |                 cytoolz.cytoolz.utils
import time:        33 |         33 |                   cytoolz.cytoolz
import time:        33 |         65 |                 cytoolz.cytoolz._signatures
import time:       266 |        266 |                 cytoolz._signatures
import time:        41 |         41 |                   cytoolz.toolz
import time:        18 |         58 |                 cytoolz.toolz.functoolz
import time:       789 |       1583 |               cytoolz.functoolz
import time:        50 |         50 |                 cytoolz.copy
import time:        39 |         39 |                   cytoolz.collections
import time:        16 |         54 |                 cytoolz.collections.abc
import time:       275 |        377 |               cytoolz.dicttoolz
import time:        51 |         51 |                 cytoolz.itertools
import time:       265 |        316 |               cytoolz.recipes
import time:     16880 |      16880 |                 cytoolz.curried.operator
import time:       272 |        272 |                 cytoolz.curried.exceptions
import time:       398 |      17550 |               cytoolz.curried
import time:       203 |        203 |               cytoolz._version
import time:      2916 |      54032 |             tlz._build_tlz
import time:       269 |      54301 |           tlz
import time:       396 |        396 |                   multiprocessing.process
import time:       332 |        332 |                       _compat_pickle
import time:       358 |        358 |                       _pickle
import time:       122 |        122 |                           org
import time:       184 |        305 |                         org.python
import time:        18 |        323 |                       org.python.core
import time:       961 |       1973 |                     pickle
import time:       829 |        829 |                       _socket
import time:      1763 |       2591 |                     socket
import time:       266 |        266 |                     array
import time:       352 |       5180 |                   multiprocessing.reduction
import time:       651 |       6226 |                 multiprocessing.context
import time:       304 |       6529 |               multiprocessing
import time:       314 |        314 |               multiprocessing.util
import time:       224 |        224 |                 _multiprocessing
import time:       114 |        114 |                 _winapi
import time:       497 |        834 |               multiprocessing.connection
import time:       502 |       8177 |             multiprocessing.pool
import time:       372 |        372 |             concurrent.futures.thread
import time:      2776 |       2776 |                 psutil._common
import time:       279 |        279 |                 psutil._compat
import time:       595 |        595 |                   psutil._psposix
import time:       234 |        234 |                   psutil._psutil_linux
import time:       154 |        154 |                   psutil._psutil_posix
import time:       236 |        236 |                   resource
import time:       520 |        520 |                     _ctypes
import time:       312 |        312 |                     ctypes._endian
import time:       942 |       1773 |                   ctypes
import time:      4844 |       7833 |                 psutil._pslinux
import time:      1204 |      12090 |               psutil
import time:       503 |      12593 |             dask.system
import time:       254 |      21394 |           dask.threaded
import time:       206 |        206 |               distutils
import time:       784 |        989 |             distutils.version
import time:       928 |        928 |               _hashlib
import time:       217 |        217 |               _blake2
import time:       209 |        209 |               _sha3
import time:       329 |       1682 |             hashlib
import time:       188 |        188 |             dask.context
import time:       113 |        113 |               cityhash
import time:       461 |        461 |                 xxhash._xxhash
import time:       206 |        667 |               xxhash
import time:       111 |        111 |               mmh3
import time:       177 |       1066 |             dask.hashing
import time:       319 |        319 |                 multiprocessing.queues
import time:       435 |        753 |               concurrent.futures.process
import time:       154 |        154 |                   cloudpickle.compat
import time:       845 |        845 |                   typing_extensions
import time:       920 |       1919 |                 cloudpickle.cloudpickle
import time:      9869 |       9869 |                 cloudpickle.cloudpickle_fast
import time:       314 |      12101 |               cloudpickle
import time:       609 |        609 |               dask.optimization
import time:       159 |        159 |                   __pypy__
import time:       423 |        423 |                   tblib.cpython
import time:       841 |       1422 |                 tblib
import time:       210 |       1631 |               tblib.pickling_support
import time:       658 |      15750 |             dask.multiprocessing
import time:       803 |      20476 |           dask.base
import time:       913 |        913 |           dask.highlevelgraph
import time:      3553 |     101397 |         dask.delayed
import time:       243 |        243 |         dask._version
import time:       368 |     164095 |       dask
import time:       286 |        286 |             numpy._globals
import time:       177 |        177 |             numpy.__config__
import time:       173 |        173 |             numpy.version
import time:       123 |        123 |             numpy._distributor_init
import time:     13948 |      13948 |                   numpy.core._multiarray_umath
import time:       314 |        314 |                       numpy.compat._inspect
import time:       128 |        128 |                             nt
import time:       131 |        131 |                             nt
import time:        79 |         79 |                             nt
import time:        74 |         74 |                             nt
import time:       352 |        763 |                           ntpath
import time:       164 |        164 |                             urllib
import time:      1238 |       1402 |                           urllib.parse
import time:      1168 |       3332 |                         pathlib
import time:       111 |        111 |                         pickle5
import time:       238 |       3681 |                       numpy.compat.py3k
import time:       408 |       4401 |                     numpy.compat
import time:        44 |       4444 |                   numpy.compat._inspect
import time:       459 |      18850 |                 numpy.core.overrides
import time:      1854 |      20703 |               numpy.core.multiarray
import time:       331 |        331 |               numpy.core.umath
import time:       209 |        209 |                 numpy.core._string_helpers
import time:       375 |        375 |                   numpy.core._dtype
import time:       442 |        817 |                 numpy.core._type_aliases
import time:       463 |       1488 |               numpy.core.numerictypes
import time:       695 |        695 |                   numpy.core._asarray
import time:       388 |        388 |                       numpy.core._exceptions
import time:       400 |        788 |                     numpy.core._methods
import time:      4099 |       4886 |                   numpy.core.fromnumeric
import time:       828 |       6407 |                 numpy.core.shape_base
import time:       387 |        387 |                 numpy.core._ufunc_config
import time:      1046 |       1046 |                 numpy.core.arrayprint
import time:      2663 |      10501 |               numpy.core.numeric
import time:      3757 |       3757 |               numpy.core.defchararray
import time:       592 |        592 |               numpy.core.records
import time:       262 |        262 |               numpy.core.memmap
import time:       499 |        499 |               numpy.core.function_base
import time:      1293 |       1293 |               numpy.core.machar
import time:       490 |        490 |               numpy.core.getlimits
import time:       574 |        574 |               numpy.core.einsumfunc
import time:       381 |        381 |                 numpy.core._multiarray_tests
import time:      1290 |       1671 |               numpy.core._add_newdocs
import time:       460 |        460 |               numpy.core._add_newdocs_scalars
import time:       171 |        171 |               numpy.core._dtype_ctypes
import time:       997 |        997 |               numpy.core._internal
import time:       223 |        223 |               numpy._pytesttester
import time:       779 |      44783 |             numpy.core
import time:       323 |        323 |               numpy.lib.mixins
import time:       426 |        426 |                   numpy.lib.ufunclike
import time:       927 |       1353 |                 numpy.lib.type_check
import time:       748 |       2100 |               numpy.lib.scimath
import time:      1045 |       1045 |                         numpy.lib.twodim_base
import time:       322 |        322 |                         numpy.linalg.lapack_lite
import time:       320 |        320 |                         numpy.linalg._umath_linalg
import time:      2417 |       4102 |                       numpy.linalg.linalg
import time:       182 |       4284 |                     numpy.linalg
import time:       368 |       4651 |                   numpy.matrixlib.defmatrix
import time:       175 |       4825 |                 numpy.matrixlib
import time:       614 |        614 |                   numpy.lib.histograms
import time:      2594 |       3208 |                 numpy.lib.function_base
import time:       485 |        485 |                 numpy.lib.stride_tricks
import time:       792 |       9309 |               numpy.lib.index_tricks
import time:      1255 |       1255 |               numpy.lib.nanfunctions
import time:      1124 |       1124 |               numpy.lib.shape_base
import time:      1667 |       1667 |               numpy.lib.polynomial
import time:       863 |        863 |               numpy.lib.utils
import time:       779 |        779 |               numpy.lib.arraysetops
import time:       268 |        268 |                 numpy.lib.format
import time:       359 |        359 |                 numpy.lib._datasource
import time:       386 |        386 |                 numpy.lib._iotools
import time:      1008 |       2020 |               numpy.lib.npyio
import time:       198 |        198 |               numpy.lib.arrayterator
import time:       339 |        339 |               numpy.lib.arraypad
import time:       185 |        185 |               numpy.lib._version
import time:       474 |      20629 |             numpy.lib
import time:       223 |        223 |                 numpy.fft._pocketfft_internal
import time:      2023 |       2246 |               numpy.fft._pocketfft
import time:       342 |        342 |               numpy.fft.helper
import time:       227 |       2814 |             numpy.fft
import time:       402 |        402 |                 numpy.polynomial.polyutils
import time:       417 |        417 |                 numpy.polynomial._polybase
import time:       631 |       1449 |               numpy.polynomial.polynomial
import time:       396 |        396 |               numpy.polynomial.chebyshev
import time:       369 |        369 |               numpy.polynomial.legendre
import time:       352 |        352 |               numpy.polynomial.hermite
import time:       325 |        325 |               numpy.polynomial.hermite_e
import time:       344 |        344 |               numpy.polynomial.laguerre
import time:       210 |       3441 |             numpy.polynomial
import time:       132 |        132 |                       backports_abc
import time:       678 |        810 |                     numpy.random._common
import time:       285 |        285 |                       hmac
import time:       218 |        502 |                     secrets
import time:       651 |       1962 |                   numpy.random.bit_generator
import time:       294 |        294 |                   numpy.random._bounded_integers
import time:       254 |        254 |                   numpy.random._mt19937
import time:      1183 |       3692 |                 numpy.random.mtrand
import time:       268 |        268 |                 numpy.random._philox
import time:       236 |        236 |                 numpy.random._pcg64
import time:       235 |        235 |                 numpy.random._sfc64
import time:       784 |        784 |                 numpy.random._generator
import time:       277 |       5491 |               numpy.random._pickle
import time:       270 |       5760 |             numpy.random
import time:       520 |        520 |             numpy.ctypeslib
import time:      3067 |       3067 |               numpy.ma.core
import time:      1284 |       1284 |               numpy.ma.extras
import time:       295 |       4646 |             numpy.ma
import time:      2087 |      85433 |           numpy
import time:       320 |        320 |                 _csv
import time:       484 |        804 |               csv
import time:       201 |        201 |               email
import time:       953 |        953 |               zipfile
import time:      1034 |       2990 |             importlib.metadata
import time:       508 |        508 |             fsspec.caching
import time:       150 |        150 |             fsspec._version
import time:       396 |        396 |                 gzip
import time:       224 |        224 |                 fsspec.utils
import time:       271 |        271 |                   fsspec.config
import time:       208 |        208 |                   fsspec.dircache
import time:       200 |        200 |                   fsspec.transaction
import time:      2765 |       3443 |                 fsspec.spec
import time:        97 |         97 |                 lzmaffi
import time:        88 |         88 |                 snappy
import time:        79 |         79 |                   lz4
import time:        17 |         96 |                 lz4.frame
import time:        78 |         78 |                 zstandard
import time:       307 |       4727 |               fsspec.compression
import time:       255 |        255 |               fsspec.registry
import time:       343 |       5324 |             fsspec.core
import time:       258 |        258 |             fsspec.mapping
import time:     32755 |      41982 |           fsspec
import time:       716 |        716 |           tlz.curried
import time:       212 |        212 |             dask.compatibility
import time:      1313 |       1525 |           dask.blockwise
import time:       326 |        326 |           dask.sizeof
import time:       875 |        875 |             dask.array.numpy_compat
import time:       327 |       1201 |           dask.array.chunk
import time:       125 |        125 |             cupy
import time:        90 |         90 |                 cupyx
import time:        14 |        104 |               cupyx.scipy
import time:        15 |        119 |             cupyx.scipy.sparse
import time:        97 |         97 |             sparse
import time:       288 |        288 |                     scipy._lib._testutils
import time:       237 |        524 |                   scipy._lib
import time:       191 |        715 |                 scipy._lib.deprecation
import time:       276 |        276 |                 scipy.__config__
import time:       137 |        137 |                 scipy.version
import time:       129 |        129 |                 scipy._distributor_init
import time:      2650 |       2650 |                 scipy._lib._pep440
import time:       483 |        483 |                   scipy._lib._ccallback_c
import time:       369 |        852 |                 scipy._lib._ccallback
import time:      3276 |       8031 |               scipy
import time:       730 |        730 |                   scipy._lib._util
import time:       267 |        996 |                 scipy.sparse.sputils
import time:       649 |       1645 |               scipy.sparse.base
import time:      3457 |       3457 |                 scipy.sparse._sparsetools
import time:       486 |        486 |                   scipy.sparse.data
import time:       290 |        290 |                   scipy.sparse.dia
import time:       276 |        276 |                   scipy.sparse._index
import time:       751 |       1800 |                 scipy.sparse.compressed
import time:       468 |       5724 |               scipy.sparse.csr
import time:       231 |        231 |               scipy.sparse.csc
import time:       863 |        863 |                 scipy.sparse._csparsetools
import time:       417 |       1279 |               scipy.sparse.lil
import time:       399 |        399 |               scipy.sparse.dok
import time:      1133 |       1133 |               scipy.sparse.coo
import time:       438 |        438 |               scipy.sparse.bsr
import time:       286 |        286 |               scipy.sparse.construct
import time:       157 |        157 |               scipy.sparse.extract
import time:       162 |        162 |               scipy.sparse._matrix_io
import time:       234 |        234 |                 scipy.sparse.csgraph._laplacian
import time:       286 |        286 |                     scipy.sparse.csgraph._tools
import time:       182 |        468 |                   scipy.sparse.csgraph._validation
import time:       499 |        967 |                 scipy.sparse.csgraph._shortest_path
import time:       252 |        252 |                 scipy.sparse.csgraph._traversal
import time:       401 |        401 |                 scipy.sparse.csgraph._min_spanning_tree
import time:       479 |        479 |                 scipy.sparse.csgraph._flow
import time:       338 |        338 |                 scipy.sparse.csgraph._matching
import time:       385 |        385 |                 scipy.sparse.csgraph._reordering
import time:       293 |       3344 |               scipy.sparse.csgraph
import time:       822 |      23646 |             scipy.sparse
import time:       229 |      24214 |           dask.array.chunk_types
import time:       753 |        753 |           dask.array.slicing
import time:       305 |        305 |           dask.array.optimization
import time:       254 |        254 |           dask.array.blockwise
import time:       956 |        956 |             difflib
import time:       388 |       1344 |           dask.array.utils
import time:      4205 |     162252 |         dask.array.core
import time:       279 |     162530 |       dask.array.backends
import time:       120 |        120 |                   uarray
import time:       465 |        465 |                       scipy._lib._uarray._uarray
import time:       570 |       1035 |                     scipy._lib._uarray._backend
import time:       169 |       1203 |                   scipy._lib._uarray
import time:       169 |       1491 |                 scipy._lib.uarray
import time:      1009 |       2500 |               scipy.fft._basic
import time:       506 |        506 |               scipy.fft._realtransforms
import time:       676 |        676 |                     scipy.fft._pocketfft.pypocketfft
import time:       428 |        428 |                     scipy.fft._pocketfft.helper
import time:       903 |       2006 |                   scipy.fft._pocketfft.basic
import time:       191 |        191 |                   scipy.fft._pocketfft.realtransforms
import time:       199 |       2394 |                 scipy.fft._pocketfft
import time:       183 |       2576 |               scipy.fft._helper
import time:       213 |        213 |               scipy.fft._backend
import time:       297 |       6090 |             scipy.fft
import time:       159 |        159 |             scipy.fftpack.helper
import time:       232 |       6480 |           scipy.fftpack.basic
import time:      1913 |       1913 |             scipy.fftpack.convolve
import time:       312 |       2225 |           scipy.fftpack.pseudo_diffs
import time:       439 |        439 |           scipy.fftpack.realtransforms
import time:       246 |       9388 |         scipy.fftpack
import time:     16234 |      16234 |           dask.array.ufunc
import time:       613 |        613 |             dask.layers
import time:       773 |       1385 |           dask.array.wrap
import time:      2713 |      20331 |         dask.array.creation
import time:      1302 |      31020 |       dask.array.fft
import time:      1160 |       1160 |           dask.array.overlap
import time:       193 |       1353 |         dask.array.lib.stride_tricks
import time:       202 |       1554 |       dask.array.lib
import time:      6477 |       6477 |         dask.array.random
import time:       920 |       7397 |       dask.array.linalg
import time:      1672 |       1672 |           dask.array.einsumfuncs
import time:     13998 |      15670 |         dask.array.routines
import time:      3819 |      19488 |       dask.array.ma
import time:       474 |        474 |       dask.array.gufunc
import time:       247 |        247 |       dask.array.percentile
import time:       478 |        478 |       dask.array.rechunk
import time:      8716 |       8716 |       dask.array.reductions
import time:       352 |        352 |       dask.array.reshape
import time:       187 |        187 |       dask.array.tiledb_io
import time:       594 |     397126 |     dask.array
import time:        68 |         68 |             cftime.numpy
import time:        41 |         41 |             cftime.re
import time:        32 |         32 |             cftime.sys
import time:        29 |         29 |             cftime.time
import time:        29 |         29 |             cftime.datetime
import time:        28 |         28 |             cftime.datetime
import time:        30 |         30 |             cftime.time
import time:        27 |         27 |             cftime.warnings
import time:        28 |         28 |             cftime.itertools
import time:      2205 |       2512 |           cftime._cftime
import time:       298 |       2810 |         cftime
import time:      1477 |       1477 |           iris._lazy_data
import time:       615 |        615 |               cf_units.config
import time:       235 |        235 |               cf_units.util
import time:        68 |         68 |                 cf_units.numpy
import time:       758 |        826 |               cf_units._udunits2
import time:       155 |        155 |               cf_units._version
import time:      8312 |      10141 |             cf_units
import time:       615 |        615 |             iris._deprecation
import time:       628 |        628 |             iris.exceptions
import time:      1391 |      12774 |           iris.util
import time:      1103 |      15353 |         iris._data_manager
import time:      1401 |       1401 |             iris.common.lenient
import time:      2738 |       2738 |             iris.common.metadata
import time:      4505 |       4505 |               iris.std_names
import time:      1334 |       5838 |             iris.common.mixin
import time:      7293 |       7293 |             iris.common.resolve
import time:       787 |      18055 |           iris.common
import time:      1299 |      19354 |         iris.aux_factory
import time:       761 |        761 |         iris.time
import time:      2058 |      40333 |       iris.coords
import time:      1987 |      42319 |     iris._concatenate
import time:      1138 |       1138 |     iris._constraints
import time:      3063 |       3063 |     iris._merge
import time:       260 |        260 |             scipy.special.sf_error
import time:       253 |        253 |               scipy.special._ufuncs_cxx
import time:       765 |       1017 |             scipy.special._ufuncs
import time:       271 |        271 |               scipy.special.specfun
import time:       453 |        453 |                       scipy.linalg._fblas
import time:        53 |         53 |                       scipy.linalg._cblas
import time:        37 |         37 |                       scipy.linalg._fblas_64
import time:       415 |        957 |                     scipy.linalg.blas
import time:       975 |        975 |                       scipy.linalg._flapack
import time:        53 |         53 |                       scipy.linalg._clapack
import time:        37 |         37 |                       scipy.linalg._flapack_64
import time:      1310 |       2374 |                     scipy.linalg.lapack
import time:       284 |       3613 |                   scipy.linalg.misc
import time:       268 |        268 |                       scipy.linalg._flinalg
import time:       184 |        452 |                     scipy.linalg.flinalg
import time:       398 |        398 |                     scipy.linalg.decomp
import time:       202 |        202 |                     scipy.linalg.decomp_svd
import time:       355 |        355 |                     scipy.linalg._solve_toeplitz
import time:       465 |       1870 |                   scipy.linalg.basic
import time:       183 |        183 |                   scipy.linalg.decomp_lu
import time:       290 |        290 |                   scipy.linalg._decomp_ldl
import time:       182 |        182 |                   scipy.linalg.decomp_cholesky
import time:       194 |        194 |                   scipy.linalg.decomp_qr
import time:       193 |        193 |                   scipy.linalg._decomp_qz
import time:       299 |        299 |                   scipy.linalg.decomp_schur
import time:       160 |        160 |                   scipy.linalg._decomp_polar
import time:       298 |        298 |                     scipy.linalg.special_matrices
import time:       199 |        199 |                     scipy.linalg._expm_frechet
import time:       363 |        363 |                       scipy.linalg._matfuncs_sqrtm_triu
import time:       213 |        575 |                     scipy.linalg._matfuncs_sqrtm
import time:       265 |       1336 |                   scipy.linalg.matfuncs
import time:       259 |        259 |                   scipy.linalg._solvers
import time:       146 |        146 |                   scipy.linalg._procrustes
import time:       493 |        493 |                     scipy.linalg.cython_blas
import time:      1495 |       1495 |                     scipy.linalg.cython_lapack
import time:       444 |       2432 |                   scipy.linalg._decomp_update
import time:       210 |        210 |                   scipy.linalg._sketches
import time:       195 |        195 |                   scipy.linalg._decomp_cossin
import time:       438 |      11992 |                 scipy.linalg
import time:       491 |      12482 |               scipy.special.orthogonal
import time:       191 |        191 |               scipy.special._comb
import time:       474 |      13416 |             scipy.special._basic
import time:       179 |        179 |             scipy.special._logsumexp
import time:       157 |        157 |             scipy.special.spfun_stats
import time:       247 |        247 |               scipy.special._ellip_harm_2
import time:       179 |        426 |             scipy.special._ellip_harm
import time:       144 |        144 |             scipy.special._lambertw
import time:       174 |        174 |             scipy.special._spherical_bessel
import time:       462 |      16231 |           scipy.special
import time:       219 |        219 |               scipy.interpolate._fitpack
import time:       259 |        259 |               scipy.interpolate.dfitpack
import time:       349 |        827 |             scipy.interpolate._fitpack_impl
import time:       472 |        472 |               scipy.interpolate._bspl
import time:       415 |        887 |             scipy.interpolate._bsplines
import time:       304 |       2017 |           scipy.interpolate.fitpack
import time:       313 |        313 |           scipy.interpolate.polyint
import time:       333 |        333 |           scipy.interpolate._ppoly
import time:       585 |        585 |           scipy.interpolate.fitpack2
import time:       756 |        756 |                   scipy.spatial.ckdtree
import time:       465 |       1220 |                 scipy.spatial.kdtree
import time:       243 |        243 |                   scipy._lib.messagestream
import time:       734 |        976 |                 scipy.spatial.qhull
import time:       286 |        286 |                   scipy.spatial._voronoi
import time:       297 |        583 |                 scipy.spatial._spherical_voronoi
import time:      3273 |       3273 |                   scipy._lib.decorator
import time:       775 |       4048 |                 scipy.spatial._plotutils
import time:       194 |        194 |                 scipy.spatial._procrustes
import time:       287 |        287 |                     scipy.spatial._distance_wrap
import time:       307 |        307 |                     scipy.spatial._hausdorff
import time:       793 |       1386 |                   scipy.spatial.distance
import time:       181 |       1566 |                 scipy.spatial._geometric_slerp
import time:       138 |        138 |                     backports_abc
import time:      4703 |       4703 |                         scipy.constants.codata
import time:       402 |        402 |                         scipy.constants.constants
import time:      1256 |       6360 |                       scipy.constants
import time:       192 |       6551 |                     scipy.spatial.transform._rotation_groups
import time:       786 |       7474 |                   scipy.spatial.transform.rotation
import time:       313 |        313 |                   scipy.spatial.transform._rotation_spline
import time:       220 |       8005 |                 scipy.spatial.transform
import time:       348 |      16936 |               scipy.spatial
import time:        29 |      16965 |             scipy.spatial.qhull
import time:       572 |      17536 |           scipy.interpolate.interpnd
import time:       815 |      37828 |         scipy.interpolate.interpolate
import time:       325 |        325 |         scipy.interpolate.rbf
import time:       341 |        341 |         scipy.interpolate._cubic
import time:       211 |        211 |         scipy.interpolate.ndgriddata
import time:       172 |        172 |         scipy.interpolate._pade
import time:       394 |      39268 |       scipy.interpolate
import time:       174 |        174 |                 scipy.ndimage._ni_support
import time:       395 |        395 |                 scipy.ndimage._nd_image
import time:       212 |        212 |                   scipy._lib.doccer
import time:       303 |        515 |                 scipy.ndimage._ni_docstrings
import time:      2401 |       3484 |               scipy.ndimage.filters
import time:       215 |        215 |               scipy.ndimage.fourier
import time:       934 |        934 |               scipy.ndimage.interpolation
import time:       694 |        694 |                 scipy.ndimage._ni_label
import time:       391 |        391 |                 scipy.ndimage.morphology
import time:       505 |       1589 |               scipy.ndimage.measurements
import time:       317 |       6538 |             scipy.ndimage
import time:       223 |        223 |                 scipy.stats._distr_params
import time:       303 |        303 |                       scipy.optimize.minpack2
import time:       318 |        621 |                     scipy.optimize.linesearch
import time:       341 |        341 |                             scipy.sparse.linalg.isolve._iterative
import time:       530 |        530 |                             scipy.sparse.linalg.interface
import time:       177 |        177 |                             scipy.sparse.linalg.isolve.utils
import time:       215 |        215 |                             scipy._lib._threadsafety
import time:      1357 |       2619 |                           scipy.sparse.linalg.isolve.iterative
import time:       216 |        216 |                           scipy.sparse.linalg.isolve.minres
import time:       201 |        201 |                             scipy.sparse.linalg.isolve._gcrotmk
import time:       190 |        391 |                           scipy.sparse.linalg.isolve.lgmres
import time:       230 |        230 |                           scipy.sparse.linalg.isolve.lsqr
import time:       194 |        194 |                           scipy.sparse.linalg.isolve.lsmr
import time:       284 |       3931 |                         scipy.sparse.linalg.isolve
import time:       344 |        344 |                             scipy.sparse.linalg.dsolve._superlu
import time:       124 |        124 |                               scikits
import time:        20 |        143 |                             scikits.umfpack
import time:       388 |        874 |                           scipy.sparse.linalg.dsolve.linsolve
import time:       180 |        180 |                           scipy.sparse.linalg.dsolve._add_newdocs
import time:       215 |       1268 |                         scipy.sparse.linalg.dsolve
import time:       448 |        448 |                               scipy.sparse.linalg.eigen.arpack._arpack
import time:       282 |        282 |                                 scipy.sparse.linalg.eigen.lobpcg.lobpcg
import time:       152 |        434 |                               scipy.sparse.linalg.eigen.lobpcg
import time:       815 |       1695 |                             scipy.sparse.linalg.eigen.arpack.arpack
import time:       204 |       1899 |                           scipy.sparse.linalg.eigen.arpack
import time:       180 |       2078 |                         scipy.sparse.linalg.eigen
import time:       284 |        284 |                           scipy.sparse.linalg._expm_multiply
import time:       411 |        695 |                         scipy.sparse.linalg.matfuncs
import time:       240 |        240 |                         scipy.sparse.linalg._onenormest
import time:       163 |        163 |                         scipy.sparse.linalg._norm
import time:       279 |       8651 |                       scipy.sparse.linalg
import time:       543 |        543 |                       scipy.optimize._group_columns
import time:       373 |       9566 |                     scipy.optimize._numdiff
import time:       359 |        359 |                       scipy.optimize._hessian_update_strategy
import time:       370 |        728 |                     scipy.optimize._differentiable_functions
import time:       923 |      11837 |                   scipy.optimize.optimize
import time:       234 |        234 |                       scipy.optimize._trustregion
import time:       198 |        431 |                     scipy.optimize._trustregion_dogleg
import time:       185 |        185 |                     scipy.optimize._trustregion_ncg
import time:       730 |        730 |                         scipy.optimize._trlib._trlib
import time:       177 |        906 |                       scipy.optimize._trlib
import time:       147 |       1053 |                     scipy.optimize._trustregion_krylov
import time:       273 |        273 |                     scipy.optimize._trustregion_exact
import time:       347 |        347 |                                 unittest.util
import time:       361 |        707 |                               unittest.result
import time:      1649 |       1649 |                                       _ssl
import time:      4091 |       5740 |                                     ssl
import time:       362 |        362 |                                     asyncio.constants
import time:       252 |        252 |                                         asyncio.format_helpers
import time:       220 |        472 |                                       asyncio.base_futures
import time:       181 |        181 |                                       asyncio.log
import time:       326 |        978 |                                     asyncio.coroutines
import time:       277 |        277 |                                         _contextvars
import time:       193 |        469 |                                       contextvars
import time:       249 |        249 |                                       asyncio.exceptions
import time:       189 |        189 |                                         asyncio.base_tasks
import time:       378 |        567 |                                       _asyncio
import time:       561 |       1845 |                                     asyncio.events
import time:       277 |        277 |                                     asyncio.futures
import time:       334 |        334 |                                     asyncio.protocols
import time:       313 |        313 |                                       asyncio.transports
import time:       510 |        822 |                                     asyncio.sslproto
import time:       356 |        356 |                                       asyncio.locks
import time:       490 |        490 |                                       asyncio.tasks
import time:       424 |       1269 |                                     asyncio.staggered
import time:       256 |        256 |                                     asyncio.trsock
import time:      1052 |      12930 |                                   asyncio.base_events
import time:       187 |        187 |                                   asyncio.runners
import time:       290 |        290 |                                   asyncio.queues
import time:       485 |        485 |                                   asyncio.streams
import time:       296 |        296 |                                   asyncio.subprocess
import time:       296 |        296 |                                     asyncio.base_subprocess
import time:       582 |        582 |                                     asyncio.selector_events
import time:       680 |       1556 |                                   asyncio.unix_events
import time:       307 |      16049 |                                 asyncio
import time:       449 |        449 |                                   pprint
import time:       944 |       1392 |                                 unittest.case
import time:       222 |      17662 |                               unittest.async_case
import time:       292 |        292 |                               unittest.suite
import time:       759 |        759 |                               unittest.loader
import time:       993 |        993 |                                     locale
import time:      1046 |       2039 |                                   gettext
import time:       987 |       3025 |                                 argparse
import time:       184 |        184 |                                   unittest.signals
import time:       278 |        461 |                                 unittest.runner
import time:       288 |       3774 |                               unittest.main
import time:       287 |      23479 |                             unittest
import time:       215 |        215 |                               numpy.testing._private
import time:        68 |         68 |                               gc
import time:       811 |       1093 |                             numpy.testing._private.utils
import time:       209 |        209 |                             numpy.testing._private.decorators
import time:       243 |        243 |                             numpy.testing._private.nosetester
import time:       235 |      25256 |                           numpy.testing
import time:       324 |      25579 |                         scipy.optimize._constraints
import time:       115 |        115 |                               sksparse
import time:        24 |        139 |                             sksparse.cholmod
import time:       232 |        370 |                           scipy.optimize._trustregion_constr.projections
import time:       210 |        210 |                           scipy.optimize._trustregion_constr.qp_subproblem
import time:       205 |        784 |                         scipy.optimize._trustregion_constr.equality_constrained_sqp
import time:       234 |        234 |                         scipy.optimize._trustregion_constr.canonical_constraint
import time:       347 |        347 |                         scipy.optimize._trustregion_constr.tr_interior_point
import time:       215 |        215 |                         scipy.optimize._trustregion_constr.report
import time:       434 |      27590 |                       scipy.optimize._trustregion_constr.minimize_trustregion_constr
import time:       174 |      27763 |                     scipy.optimize._trustregion_constr
import time:       362 |        362 |                       scipy.optimize._lbfgsb
import time:       292 |        653 |                     scipy.optimize.lbfgsb
import time:       179 |        179 |                       scipy.optimize.moduleTNC
import time:       254 |        433 |                     scipy.optimize.tnc
import time:       230 |        230 |                       scipy.optimize._cobyla
import time:       268 |        497 |                     scipy.optimize.cobyla
import time:       216 |        216 |                       scipy.optimize._slsqp
import time:       327 |        543 |                     scipy.optimize.slsqp
import time:       384 |      32210 |                   scipy.optimize._minimize
import time:       209 |        209 |                       scipy.optimize._minpack
import time:       418 |        418 |                             scipy.optimize._lsq.common
import time:       260 |        678 |                           scipy.optimize._lsq.trf
import time:       221 |        221 |                           scipy.optimize._lsq.dogbox
import time:       366 |       1263 |                         scipy.optimize._lsq.least_squares
import time:       336 |        336 |                             scipy.optimize._lsq.givens_elimination
import time:       217 |        553 |                           scipy.optimize._lsq.trf_linear
import time:       181 |        181 |                           scipy.optimize._lsq.bvls
import time:       230 |        963 |                         scipy.optimize._lsq.lsq_linear
import time:       184 |       2409 |                       scipy.optimize._lsq
import time:       360 |       2977 |                     scipy.optimize.minpack
import time:       254 |        254 |                     scipy.optimize._spectral
import time:      2858 |       2858 |                     scipy.optimize.nonlin
import time:       303 |       6391 |                   scipy.optimize._root
import time:       162 |        162 |                       scipy.optimize._zeros
import time:       370 |        532 |                     scipy.optimize.zeros
import time:       250 |        782 |                   scipy.optimize._root_scalar
import time:       226 |        226 |                     scipy.optimize.__nnls
import time:       202 |        428 |                   scipy.optimize._nnls
import time:       334 |        334 |                   scipy.optimize._basinhopping
import time:       152 |        152 |                         scipy.optimize._highs
import time:       868 |       1020 |                       scipy.optimize._highs._highs_wrapper
import time:       222 |        222 |                       scipy.optimize._highs._highs_constants
import time:       260 |       1501 |                     scipy.optimize._linprog_highs
import time:       297 |        297 |                               scipy.linalg._interpolative
import time:       288 |        585 |                             scipy.linalg._interpolative_backend
import time:       220 |        805 |                           scipy.linalg.interpolative
import time:       219 |       1024 |                         scipy.optimize._remove_redundancy
import time:       482 |       1505 |                       scipy.optimize._linprog_util
import time:       133 |        133 |                       sksparse
import time:        82 |         82 |                         scikits
import time:        17 |         99 |                       scikits.umfpack
import time:       340 |       2076 |                     scipy.optimize._linprog_ip
import time:       210 |        210 |                     scipy.optimize._linprog_simplex
import time:       410 |        410 |                       scipy.optimize._bglu_dense
import time:       321 |        730 |                     scipy.optimize._linprog_rs
import time:       243 |        243 |                     scipy.optimize._linprog_doc
import time:       279 |       5038 |                   scipy.optimize._linprog
import time:       214 |        214 |                     scipy.optimize._lsap_module
import time:       169 |        382 |                   scipy.optimize._lsap
import time:       407 |        407 |                   scipy.optimize._differentialevolution
import time:       175 |        175 |                     scipy.optimize._shgo_lib
import time:       229 |        229 |                     scipy.optimize._shgo_lib.sobol_seq
import time:       408 |        408 |                     scipy.optimize._shgo_lib.triangulation
import time:       459 |       1270 |                   scipy.optimize._shgo
import time:       342 |        342 |                   scipy.optimize._dual_annealing
import time:       249 |        249 |                   scipy.optimize._qap
import time:       415 |      60079 |                 scipy.optimize
import time:       414 |        414 |                   scipy.integrate._quadrature
import time:       250 |        250 |                     scipy.integrate._odepack
import time:       215 |        464 |                   scipy.integrate.odepack
import time:       247 |        247 |                     scipy.integrate._quadpack
import time:       362 |        609 |                   scipy.integrate.quadpack
import time:       258 |        258 |                     scipy.integrate.vode
import time:       223 |        223 |                     scipy.integrate._dop
import time:       223 |        223 |                     scipy.integrate.lsoda
import time:       654 |       1357 |                   scipy.integrate._ode
import time:       362 |        362 |                   scipy.integrate._bvp
import time:       253 |        253 |                         scipy.integrate._ivp.common
import time:       317 |        317 |                         scipy.integrate._ivp.base
import time:       311 |        880 |                       scipy.integrate._ivp.bdf
import time:       420 |        420 |                       scipy.integrate._ivp.radau
import time:       233 |        233 |                         scipy.integrate._ivp.dop853_coefficients
import time:       399 |        632 |                       scipy.integrate._ivp.rk
import time:       233 |        233 |                       scipy.integrate._ivp.lsoda
import time:       350 |       2513 |                     scipy.integrate._ivp.ivp
import time:       204 |       2716 |                   scipy.integrate._ivp
import time:       325 |        325 |                   scipy.integrate._quad_vec
import time:       303 |       6547 |                 scipy.integrate
import time:       240 |        240 |                   scipy.misc.doccer
import time:       182 |        182 |                   scipy.misc.common
import time:       215 |        636 |                 scipy.misc
import time:       162 |        162 |                 scipy.stats._constants
import time:      1506 |      69150 |               scipy.stats._distn_infrastructure
import time:       947 |        947 |                   scipy.special.cython_special
import time:       670 |       1616 |                 scipy.stats._stats
import time:       228 |        228 |                 scipy.stats._rvs_sampling
import time:       239 |        239 |                 scipy.stats._tukeylambda_stats
import time:       310 |        310 |                 scipy.stats._ksstats
import time:     81443 |      83834 |               scipy.stats._continuous_distns
import time:     10337 |      10337 |               scipy.stats._discrete_distns
import time:       370 |     163689 |             scipy.stats.distributions
import time:       215 |        215 |                 scipy._lib._bunch
import time:       457 |        672 |               scipy.stats._stats_mstats_common
import time:      1873 |       2545 |             scipy.stats.mstats_basic
import time:       221 |        221 |               scipy.stats._wilcoxon_data
import time:       395 |        616 |             scipy.stats._hypotests
import time:      3604 |     176989 |           scipy.stats.stats
import time:       328 |        328 |             scipy.stats.statlib
import time:       184 |        184 |             scipy.stats.contingency
import time:      1537 |       2048 |           scipy.stats.morestats
import time:       495 |        495 |           scipy.stats._binned_statistic
import time:       277 |        277 |             scipy.stats.mvn
import time:       302 |        579 |           scipy.stats.kde
import time:       261 |        261 |             scipy.stats.mstats_extras
import time:       187 |        448 |           scipy.stats.mstats
import time:      3808 |       3808 |           scipy.stats._multivariate
import time:       460 |     184823 |         scipy.stats
import time:        22 |     184845 |       scipy.stats.mstats
import time:       832 |        832 |           iris.analysis._scipy_interpolate
import time:       201 |        201 |                 cartopy._version
import time:        64 |         64 |                 cartopy.siteconfig
import time:        95 |         95 |                 cartopy_userconfig
import time:       230 |        230 |                     shapely
import time:       193 |        193 |                       shapely.affinity
import time:       242 |        242 |                           ctypes.util
import time:       254 |        254 |                             shapely.errors
import time:       279 |        532 |                           shapely.ctypes_declarations
import time:     13498 |      14271 |                         shapely.geos
import time:       410 |        410 |                         shapely.topology
import time:       273 |      14953 |                       shapely.coords
import time:       165 |        165 |                         shapely.algorithms
import time:       210 |        210 |                         shapely.algorithms.cga
import time:       193 |        193 |                         shapely.linref
import time:       170 |        170 |                         shapely.predicates
import time:       504 |       1240 |                       shapely.impl
import time:       809 |      17193 |                     shapely.geometry.base
import time:       211 |        211 |                         shapely.geometry.proxy
import time:       264 |        474 |                       shapely.geometry.point
import time:       256 |        256 |                       shapely.geometry.linestring
import time:       394 |        394 |                       shapely.geometry.polygon
import time:       383 |        383 |                       shapely.geometry.multipoint
import time:       258 |        258 |                       shapely.geometry.multilinestring
import time:       242 |        242 |                       shapely.geometry.multipolygon
import time:       183 |        183 |                       shapely.geometry.collection
import time:       294 |       2481 |                     shapely.geometry.geo
import time:       157 |        157 |                         backports_abc
import time:        45 |         45 |                         shapely.speedups.ctypes
import time:        47 |         47 |                         shapely.speedups.logging
import time:        32 |         32 |                           shapely.speedups.shapely
import time:        16 |         47 |                         shapely.speedups.shapely.geos
import time:        33 |         33 |                           shapely.speedups.shapely
import time:        14 |         46 |                         shapely.speedups.shapely.geometry
import time:        32 |         32 |                             shapely.speedups.shapely
import time:        15 |         47 |                           shapely.speedups.shapely.geometry
import time:        11 |         57 |                         shapely.speedups.shapely.geometry.base
import time:        29 |         29 |                           shapely.speedups.shapely
import time:        13 |         42 |                         shapely.speedups.shapely.errors
import time:        33 |         33 |                         shapely.speedups.numpy
import time:       683 |       1154 |                       shapely.speedups._speedups
import time:       271 |       1424 |                     shapely.speedups
import time:       315 |      21642 |                   shapely.geometry
import time:       228 |        228 |                   shapely.prepared
import time:       190 |        190 |                     backports_abc
import time:        45 |         45 |                     cartopy.collections
import time:        35 |         35 |                     cartopy.re
import time:        33 |         33 |                     cartopy.warnings
import time:        30 |         30 |                     cartopy.numpy
import time:      5623 |       5953 |                   cartopy._crs
import time:        59 |         59 |                       cartopy.cartopy
import time:        24 |         82 |                     cartopy.cartopy._crs
import time:        35 |         35 |                       cartopy.shapely
import time:        16 |         51 |                     cartopy.shapely.geometry
import time:        34 |         34 |                       cartopy.shapely
import time:        14 |         48 |                     cartopy.shapely.geos
import time:       680 |        860 |                   cartopy.trace
import time:      3000 |      31681 |                 cartopy.crs
import time:     20366 |      20366 |                           http
import time:       521 |        521 |                               email.errors
import time:      1099 |       1099 |                                   email.quoprimime
import time:       190 |        190 |                                   email.base64mime
import time:       287 |        287 |                                       quopri
import time:       178 |        465 |                                     email.encoders
import time:       270 |        734 |                                   email.charset
import time:      1009 |       3030 |                                 email.header
import time:       740 |        740 |                                     calendar
import time:       959 |       1698 |                                   email._parseaddr
import time:       815 |       2512 |                                 email.utils
import time:       419 |       5961 |                               email._policybase
import time:       851 |       7332 |                             email.feedparser
import time:       384 |       7715 |                           email.parser
import time:       349 |        349 |                             uu
import time:      2954 |       2954 |                             email._encoded_words
import time:       246 |        246 |                             email.iterators
import time:      1204 |       4752 |                           email.message
import time:      1602 |      34433 |                         http.client
import time:       236 |        236 |                           urllib.response
import time:       286 |        522 |                         urllib.error
import time:      1895 |      36849 |                       urllib.request
import time:       563 |      37411 |                     cartopy.io
import time:       836 |        836 |                     shapefile
import time:       119 |        119 |                     fiona
import time:       572 |      38937 |                   cartopy.io.shapereader
import time:      1066 |      40002 |                 cartopy.feature
import time:      1543 |      73584 |               cartopy
import time:       105 |        105 |                 pykdtree
import time:        26 |        130 |               pykdtree.kdtree
import time:       284 |      73998 |             cartopy.img_transform
import time:      1511 |       1511 |             iris.coord_systems
import time:      1206 |       1206 |             iris.analysis._grid_angles
import time:      1550 |      78262 |           iris.analysis.cartography
import time:      1132 |      80225 |         iris.analysis._interpolation
import time:       711 |        711 |           iris.experimental
import time:      1497 |       1497 |           iris.analysis._regrid
import time:      1919 |       4125 |         iris.experimental.regrid
import time:      1335 |      85684 |       iris.analysis._area_weighted
import time:     11519 |     321314 |     iris.analysis
import time:      1621 |       1621 |     iris.analysis.maths
import time:      3444 |     775314 |   iris.cube
import time:      2736 |       2736 |       iris.io
import time:      2672 |       5407 |     iris.io.format_picker
import time:      2191 |       2191 |     iris.fileformats.abf
import time:      8986 |       8986 |           iris.fileformats._ff_cross_references
import time:      2372 |       2372 |                 iris.fileformats.um_cf_map
import time:      1685 |       4057 |               iris.fileformats.rules
import time:       884 |        884 |               iris.fileformats._pp_lbproc_pairs
import time:      1400 |       6340 |             iris.fileformats.pp_load_rules
import time:      1034 |       1034 |             iris.fileformats.pp_save_rules
import time:       528 |        528 |               mo_pack._packing
import time:       260 |        788 |             mo_pack
import time:      3089 |      11249 |           iris.fileformats.pp
import time:      1445 |      21679 |         iris.fileformats._ff
import time:       836 |      22514 |       iris.fileformats.um._ff_replacement
import time:       869 |        869 |             iris.fileformats._structured_array_identification
import time:       550 |       1419 |           iris.fileformats.um._optimal_array_structuring
import time:       657 |       2075 |         iris.fileformats.um._fast_load_structured_fields
import time:      1119 |       3193 |       iris.fileformats.um._fast_load
import time:      1721 |      27427 |     iris.fileformats.um
import time:       492 |        492 |     iris.fileformats.name
import time:       231 |        231 |           backports_abc
import time:       257 |        257 |             getopt
import time:       468 |        724 |           netCDF4.utils
import time:        48 |         48 |           netCDF4.sys
import time:        37 |         37 |           netCDF4.posixpath
import time:        33 |         33 |           netCDF4.cftime
import time:        32 |         32 |           netCDF4.numpy
import time:        30 |         30 |           netCDF4.weakref
import time:        30 |         30 |           netCDF4.warnings
import time:        29 |         29 |           netCDF4.subprocess
import time:        30 |         30 |           netCDF4.pathlib
import time:        42 |         42 |           netCDF4.glob
import time:        31 |         31 |           netCDF4.numpy
import time:     11109 |      12402 |         netCDF4._netCDF4
import time:       300 |      12701 |       netCDF4
import time:       231 |        231 |       pyke
import time:       455 |        455 |         imp
import time:       225 |        225 |           pyke.pattern
import time:       200 |        200 |           pyke.unique
import time:       308 |        732 |         pyke.contexts
import time:       615 |       1801 |       pyke.knowledge_engine
import time:      5952 |       5952 |       iris.fileformats.cf
import time:       782 |        782 |       iris.fileformats._pyke_rules
import time:      2669 |      24134 |     iris.fileformats.netcdf
import time:      1207 |       1207 |       iris.fileformats.nimrod_load_rules
import time:      1282 |       2489 |     iris.fileformats.nimrod
import time:      1774 |      63911 |   iris.fileformats
import time:      1127 |       1127 |   iris_sample_data
import time:        65 |         65 |   iris.site_config
import time:      1366 |     857046 | iris
To an ~120ms "import iris" time with this PR
> python -X importtime -c "import iris"
import time: self [us] | cumulative | imported package
import time:       341 |        341 |   _io
import time:        55 |         55 |   marshal
import time:       622 |        622 |   posix
import time:       642 |       1659 | _frozen_importlib_external
import time:       143 |        143 |   time
import time:       454 |        596 | zipimport
import time:        62 |         62 |     _codecs
import time:       675 |        736 |   codecs
import time:       496 |        496 |   encodings.aliases
import time:       802 |       2033 | encodings
import time:       220 |        220 | encodings.utf_8
import time:       200 |        200 | _signal
import time:       284 |        284 | encodings.latin_1
import time:        48 |         48 |     _abc
import time:       279 |        326 |   abc
import time:       298 |        624 | io
import time:        62 |         62 |       _stat
import time:       312 |        374 |     stat
import time:      1043 |       1043 |     _collections_abc
import time:       185 |        185 |       genericpath
import time:       276 |        461 |     posixpath
import time:       605 |       2481 |   os
import time:       203 |        203 |   _sitebuiltins
import time:        73 |         73 |     _locale
import time:       152 |        225 |   _bootlocale
import time:       268 |        268 |   types
import time:       266 |        266 |       warnings
import time:       187 |        452 |     importlib
import time:       201 |        201 |       importlib.machinery
import time:      1111 |       1311 |     importlib.abc
import time:        90 |         90 |           _operator
import time:       363 |        452 |         operator
import time:       151 |        151 |         keyword
import time:       294 |        294 |           _heapq
import time:       188 |        481 |         heapq
import time:       113 |        113 |         itertools
import time:       234 |        234 |         reprlib
import time:       968 |        968 |         _collections
import time:      1039 |       3436 |       collections
import time:        70 |         70 |         _functools
import time:       690 |        759 |       functools
import time:       693 |       4887 |     contextlib
import time:       463 |       7112 |   importlib.util
import time:       498 |        498 |   sitecustomize
import time:       125 |        125 |   usercustomize
import time:      3929 |      14838 | site
import time:       709 |        709 |       enum
import time:       186 |        186 |         _sre
import time:       375 |        375 |           sre_constants
import time:       407 |        781 |         sre_parse
import time:       339 |       1306 |       sre_compile
import time:       199 |        199 |       copyreg
import time:       554 |       2767 |     re
import time:       180 |        180 |     fnmatch
import time:       448 |       3394 |   glob
import time:       257 |        257 |     _weakrefset
import time:       649 |        905 |   threading
import time:       220 |        220 |     collections.abc
import time:       294 |        294 |       numpy._globals
import time:       165 |        165 |       numpy.__config__
import time:       156 |        156 |       numpy.version
import time:       121 |        121 |       numpy._distributor_init
import time:      1366 |       1366 |             textwrap
import time:       280 |        280 |                 math
import time:       302 |        302 |                 _datetime
import time:      1098 |       1679 |               datetime
import time:      5831 |       7509 |             numpy.core._multiarray_umath
import time:       791 |        791 |                 numpy.compat._inspect
import time:       123 |        123 |                       nt
import time:        85 |         85 |                       nt
import time:        74 |         74 |                       nt
import time:        72 |         72 |                       nt
import time:       327 |        678 |                     ntpath
import time:        75 |         75 |                     errno
import time:       162 |        162 |                       urllib
import time:      1148 |       1310 |                     urllib.parse
import time:      1115 |       3176 |                   pathlib
import time:       107 |        107 |                   pickle5
import time:       236 |        236 |                       _struct
import time:       191 |        426 |                     struct
import time:       372 |        372 |                     _compat_pickle
import time:       296 |        296 |                     _pickle
import time:       103 |        103 |                         org
import time:        17 |        119 |                       org.python
import time:        19 |        137 |                     org.python.core
import time:       888 |       2118 |                   pickle
import time:       245 |       5645 |                 numpy.compat.py3k
import time:       262 |       6697 |               numpy.compat
import time:        22 |       6719 |             numpy.compat._inspect
import time:       435 |      16028 |           numpy.core.overrides
import time:      3238 |      19266 |         numpy.core.multiarray
import time:       231 |        231 |         numpy.core.umath
import time:       476 |        476 |           numbers
import time:       195 |        195 |           numpy.core._string_helpers
import time:       211 |        211 |             numpy.core._dtype
import time:      1354 |       1565 |           numpy.core._type_aliases
import time:       381 |       2615 |         numpy.core.numerictypes
import time:       587 |        587 |             numpy.core._asarray
import time:       360 |        360 |                 numpy.core._exceptions
import time:       282 |        642 |               numpy.core._methods
import time:      3212 |       3853 |             numpy.core.fromnumeric
import time:       735 |       5174 |           numpy.core.shape_base
import time:       277 |        277 |           numpy.core._ufunc_config
import time:       872 |        872 |           numpy.core.arrayprint
import time:      2595 |       8917 |         numpy.core.numeric
import time:      3222 |       3222 |         numpy.core.defchararray
import time:       404 |        404 |         numpy.core.records
import time:       229 |        229 |         numpy.core.memmap
import time:       535 |        535 |         numpy.core.function_base
import time:       214 |        214 |         numpy.core.machar
import time:       311 |        311 |         numpy.core.getlimits
import time:       482 |        482 |         numpy.core.einsumfunc
import time:       332 |        332 |           numpy.core._multiarray_tests
import time:      1287 |       1618 |         numpy.core._add_newdocs
import time:       443 |        443 |         numpy.core._add_newdocs_scalars
import time:       161 |        161 |         numpy.core._dtype_ctypes
import time:        97 |         97 |             _ast
import time:      1016 |       1112 |           ast
import time:      2127 |       2127 |           platform
import time:       583 |        583 |             _ctypes
import time:       312 |        312 |             ctypes._endian
import time:       899 |       1793 |           ctypes
import time:       942 |       5973 |         numpy.core._internal
import time:       201 |        201 |         numpy._pytesttester
import time:       710 |      45525 |       numpy.core
import time:       398 |        398 |         numpy.lib.mixins
import time:       425 |        425 |             numpy.lib.ufunclike
import time:       890 |       1314 |           numpy.lib.type_check
import time:       770 |       2083 |         numpy.lib.scimath
import time:      1038 |       1038 |                   numpy.lib.twodim_base
import time:       303 |        303 |                   numpy.linalg.lapack_lite
import time:       305 |        305 |                   numpy.linalg._umath_linalg
import time:      2362 |       4007 |                 numpy.linalg.linalg
import time:       183 |       4189 |               numpy.linalg
import time:       437 |       4626 |             numpy.matrixlib.defmatrix
import time:       176 |       4802 |           numpy.matrixlib
import time:       527 |        527 |             numpy.lib.histograms
import time:      3334 |       3861 |           numpy.lib.function_base
import time:       484 |        484 |           numpy.lib.stride_tricks
import time:       688 |       9833 |         numpy.lib.index_tricks
import time:      1174 |       1174 |         numpy.lib.nanfunctions
import time:      1819 |       1819 |         numpy.lib.shape_base
import time:      1209 |       1209 |         numpy.lib.polynomial
import time:       757 |        757 |         numpy.lib.utils
import time:       786 |        786 |         numpy.lib.arraysetops
import time:       476 |        476 |           weakref
import time:       251 |        251 |           numpy.lib.format
import time:       385 |        385 |               zlib
import time:       276 |        276 |                 _compression
import time:       216 |        216 |                 _bz2
import time:       292 |        783 |               bz2
import time:       342 |        342 |                 _lzma
import time:       290 |        632 |               lzma
import time:        48 |         48 |               pwd
import time:       191 |        191 |               grp
import time:       822 |       2859 |             shutil
import time:       283 |       3141 |           numpy.lib._datasource
import time:       442 |        442 |           numpy.lib._iotools
import time:      1141 |       5449 |         numpy.lib.npyio
import time:       222 |        222 |         numpy.lib.arrayterator
import time:      1238 |       1238 |         numpy.lib.arraypad
import time:       213 |        213 |         numpy.lib._version
import time:       476 |      25651 |       numpy.lib
import time:       200 |        200 |           numpy.fft._pocketfft_internal
import time:      1376 |       1576 |         numpy.fft._pocketfft
import time:       347 |        347 |         numpy.fft.helper
import time:       207 |       2129 |       numpy.fft
import time:       317 |        317 |           numpy.polynomial.polyutils
import time:       474 |        474 |           numpy.polynomial._polybase
import time:       442 |       1232 |         numpy.polynomial.polynomial
import time:       386 |        386 |         numpy.polynomial.chebyshev
import time:       347 |        347 |         numpy.polynomial.legendre
import time:       342 |        342 |         numpy.polynomial.hermite
import time:       335 |        335 |         numpy.polynomial.hermite_e
import time:       402 |        402 |         numpy.polynomial.laguerre
import time:       275 |       3316 |       numpy.polynomial
import time:       138 |        138 |                 backports_abc
import time:       645 |        783 |               numpy.random._common
import time:       271 |        271 |                   binascii
import time:       993 |       1263 |                 base64
import time:       795 |        795 |                   _hashlib
import time:       216 |        216 |                     _blake2
import time:       215 |        215 |                     _sha3
import time:       377 |        807 |                   hashlib
import time:       305 |       1906 |                 hmac
import time:       184 |        184 |                     _bisect
import time:       188 |        371 |                   bisect
import time:       776 |        776 |                   _sha512
import time:       197 |        197 |                   _random
import time:       502 |       1844 |                 random
import time:       214 |       5225 |               secrets
import time:       648 |       6655 |             numpy.random.bit_generator
import time:       306 |        306 |             numpy.random._bounded_integers
import time:       248 |        248 |             numpy.random._mt19937
import time:      1061 |       8268 |           numpy.random.mtrand
import time:       277 |        277 |           numpy.random._philox
import time:       241 |        241 |           numpy.random._pcg64
import time:       214 |        214 |           numpy.random._sfc64
import time:       797 |        797 |           numpy.random._generator
import time:       277 |      10072 |         numpy.random._pickle
import time:       258 |      10329 |       numpy.random
import time:       512 |        512 |       numpy.ctypeslib
import time:       217 |        217 |                 _opcode
import time:       774 |        991 |               opcode
import time:       542 |       1532 |             dis
import time:       207 |        207 |                 token
import time:      1806 |       2012 |               tokenize
import time:       221 |       2233 |             linecache
import time:      2078 |       5842 |           inspect
import time:      3191 |       9032 |         numpy.ma.core
import time:      1352 |       1352 |         numpy.ma.extras
import time:       306 |      10688 |       numpy.ma
import time:      2319 |     101199 |     numpy
import time:      2012 |       2012 |     iris.exceptions
import time:      1021 |     104451 |   iris._constraints
import time:      2279 |       2279 |     configparser
import time:       375 |        375 |       traceback
import time:        51 |         51 |         _string
import time:       847 |        897 |       string
import time:        57 |         57 |       atexit
import time:      2086 |       3414 |     logging
import time:      1304 |       6997 |   iris.config
import time:       532 |        532 |   iris.io
import time:       407 |        407 |   iris._deprecation
import time:       250 |        250 |   iris_sample_data
import time:        54 |         54 |   iris.site_config
import time:      1013 |     117999 | iris

@bjlittle
Copy link
Member Author

bjlittle commented Jun 3, 2021

This leaves our iris import relationship graph looking like the following:

iris

If you seem suitably horrified... this doesn't include any Python Standard Library or third-party module dependencies, nor the iris.tests ... and this is the "tidied" version of our import dependencies 😱

@bjlittle
Copy link
Member Author

bjlittle commented Jun 3, 2021

@trexfeathers I'm super keen to see this branch benchmarked with ASV prior to merging, to give us all confidence that I've not caused major regressions elsewhere.

Is that easily possible? 🤔

@trexfeathers
Copy link
Contributor

Is that easily possible?

Yes.

to give us all confidence that I've not caused major regressions elsewhere

Importing has so far not been a focus of benchmark coverage, so you'll need to write the benchmarks first.

@pp-mo
Copy link
Member

pp-mo commented Jun 3, 2021

benchmarked with ASV prior to merging

But surely it doesn't need to be done prior to merging.
ASV can do it "retrospectively" ?

@trexfeathers
Copy link
Contributor

But surely it doesn't need to be done prior to merging.
ASV can do it "retrospectively" ?

I assume @bjlittle would rather not have this polluting the commit history if it turns out to be a bad thing.

Copy link
Contributor

@trexfeathers trexfeathers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK @bjlittle, I've genuinely been through every file in the PR files dialog and here's my feedback. Also still need some benchmark results as you suggested.

I've intentionally not included isort as a PyPI or conda dependency; let's not unnecessarily add to the dependency bloat, developers can install isort easily themselves - otherwise pre-commit is your new best friend.

Please can we do the same with Black? It looks like an identical situation, and that would solve our recent pin alignment trouble (#4162).

lib/iris/analysis/_area_weighted.py Show resolved Hide resolved
lib/iris/analysis/_regrid.py Show resolved Hide resolved
lib/iris/tests/test_peak.py Show resolved Hide resolved
lib/iris/tests/test_plot.py Outdated Show resolved Hide resolved
lib/iris/tests/unit/constraints/test_NameConstraint.py Outdated Show resolved Hide resolved
lib/iris/tests/unit/fileformats/pp/test_save.py Outdated Show resolved Hide resolved
lib/iris/tests/unit/representation/test_representation.py Outdated Show resolved Hide resolved
@bjlittle
Copy link
Member Author

bjlittle commented Jun 4, 2021

@trexfeathers I made all your suggested changes, thanks ... 9 files out of 423, better than I'd thought. I'll happily take a 2% error rate 😄

Also, yes, the same principle applies to black and also flake8. But I'll make that change in a separate PR... this one is noisy enough. BTW this won't affect cirrus-ci, as it installs black and flake8 (like isort) in a task with pip, not conda 👍

@rcomer
Copy link
Member

rcomer commented Jun 7, 2021

@bjlittle looks like someone merged changes over the weekend that have given you conflicts 😞

@pp-mo
Copy link
Member

pp-mo commented Jun 14, 2021

Just to let you know I'm looking..
I really love the standardisation of import ordering.

As an aside, it's also brilliant how Black allows automatic resolution of the code edit changes.
Between them, Black + Isort are not only adding consistency, but taking quite a load off the devs.

However, I do find that putting all the 'froms' after all the 'imports' is non-obvious to the reader. Do we have to accept that ?

@trexfeathers
Copy link
Contributor

trexfeathers commented Jun 14, 2021

However, I do find that putting all the 'froms' after all the 'imports' is non-obvious to the reader. Do we have to accept that ?

I've been quite enjoying the opinionated linters telling me what I need to be comfortable with, rather than the other way round!

@pp-mo
Copy link
Member

pp-mo commented Jun 14, 2021

putting all the 'froms' after all the 'imports' is non-obvious to the reader.

Other than that, I'm 100% 👍 💯 on this.

I've looked over it all, and honestly all the code changes read better to me.
The use of deferred imports is logical, and actually reads better in virtually every case.

@bjlittle
Copy link
Member Author

bjlittle commented Jun 14, 2021

Do we have to accept that ?

@pp-mo I've really fallen in love with isort, it's just a lovely, lovely package and highly configurable.

But if we're finding it irksome that the from ... is sorted separately and afterwards, then I'm sure I can find the magic to make this inline... in fact, you've peeked my interest, so I'll have a read through the docs. I'm pretty convinced that I've seen this on my travels...

I know that we seem to be baking in a lot of infrastructure content into repos these days, but my ultimate goal is not to over egg it, but find a sweet compromise where the author and the reviewer/s focus on the code changes and forget about formatting, linting et al as much as possible i.e., automate what we need to, and no more - if you get my drift.

@bjlittle
Copy link
Member Author

bjlittle commented Jun 14, 2021

BTW I still need to benchmark this on ASV... I'm on that, and I'll share the results here 👍

@bjlittle
Copy link
Member Author

bjlittle commented Jun 14, 2021

@bjlittle looks like someone merged changes over the weekend that have given you conflicts 😞

@rcomer Yeah, thanks... this was always going to happen. This PR is ripe for going stale quick... I'll try to service it now that my head is out the nc-time-axis space 👍

@bjlittle
Copy link
Member Author

bjlittle commented Jun 14, 2021

@pp-mo I mean... like I said, just lovely... see https://pycqa.github.io/isort/docs/configuration/custom_sections_and_ordering/#controlling-how-isort-sections-from-imports

I'm going to go with this, as I agree, it's kinda more intuitive to read

@bjlittle
Copy link
Member Author

Okay, so this is fun... More circular imports to resolve, erk 😨

@bjlittle
Copy link
Member Author

BTW After adding the force_sort_within_sections = "True", I'm not seeing any major departure from the original advertised performance improvement of ~120ms e.g.,

This sample gives an ~112ms "import iris" time with this PR
> python -X importtime -c "import iris"
import time: self [us] | cumulative | imported package
import time:       256 |        256 |   _io
import time:        44 |         44 |   marshal
import time:       714 |        714 |   posix
import time:       589 |       1601 | _frozen_importlib_external
import time:       135 |        135 |   time
import time:       265 |        400 | zipimport
import time:        55 |         55 |     _codecs
import time:       473 |        528 |   codecs
import time:       425 |        425 |   encodings.aliases
import time:       604 |       1556 | encodings
import time:       226 |        226 | encodings.utf_8
import time:       192 |        192 | _signal
import time:       247 |        247 | encodings.latin_1
import time:        46 |         46 |     _abc
import time:       271 |        316 |   abc
import time:       327 |        643 | io
import time:        60 |         60 |       _stat
import time:       313 |        373 |     stat
import time:      1031 |       1031 |     _collections_abc
import time:       194 |        194 |       genericpath
import time:       233 |        426 |     posixpath
import time:       624 |       2453 |   os
import time:       223 |        223 |   _sitebuiltins
import time:        70 |         70 |     _locale
import time:       151 |        220 |   _bootlocale
import time:       273 |        273 |   types
import time:       260 |        260 |       warnings
import time:       187 |        447 |     importlib
import time:       151 |        151 |       importlib.machinery
import time:       509 |        660 |     importlib.abc
import time:        80 |         80 |           _operator
import time:       312 |        391 |         operator
import time:       143 |        143 |         keyword
import time:       278 |        278 |           _heapq
import time:       209 |        486 |         heapq
import time:       101 |        101 |         itertools
import time:       199 |        199 |         reprlib
import time:       619 |        619 |         _collections
import time:       823 |       2758 |       collections
import time:        67 |         67 |         _functools
import time:       546 |        613 |       functools
import time:       605 |       3975 |     contextlib
import time:       415 |       5495 |   importlib.util
import time:       673 |        673 |   sitecustomize
import time:        89 |         89 |   usercustomize
import time:      3684 |      13106 | site
import time:       714 |        714 |       enum
import time:       198 |        198 |         _sre
import time:       345 |        345 |           sre_constants
import time:       396 |        741 |         sre_parse
import time:       310 |       1248 |       sre_compile
import time:       200 |        200 |       copyreg
import time:       542 |       2702 |     re
import time:       180 |        180 |     fnmatch
import time:       476 |       3357 |   glob
import time:       247 |        247 |     _weakrefset
import time:       601 |        847 |   threading
import time:       192 |        192 |     collections.abc
import time:       277 |        277 |       numpy._globals
import time:       156 |        156 |       numpy.__config__
import time:       132 |        132 |       numpy.version
import time:       124 |        124 |       numpy._distributor_init
import time:      1284 |       1284 |             textwrap
import time:       256 |        256 |                 math
import time:       298 |        298 |                 _datetime
import time:       944 |       1497 |               datetime
import time:      5871 |       7368 |             numpy.core._multiarray_umath
import time:       221 |        221 |                 numpy.compat._inspect
import time:       131 |        131 |                       nt
import time:        93 |         93 |                       nt
import time:        83 |         83 |                       nt
import time:        82 |         82 |                       nt
import time:       320 |        707 |                     ntpath
import time:        75 |         75 |                     errno
import time:       170 |        170 |                       urllib
import time:      1177 |       1347 |                     urllib.parse
import time:      1243 |       3370 |                   pathlib
import time:       119 |        119 |                   pickle5
import time:       268 |        268 |                       _struct
import time:       180 |        448 |                     struct
import time:       379 |        379 |                     _compat_pickle
import time:       315 |        315 |                     _pickle
import time:       114 |        114 |                         org
import time:        19 |        132 |                       org.python
import time:        29 |        160 |                     org.python.core
import time:       917 |       2216 |                   pickle
import time:       221 |       5926 |                 numpy.compat.py3k
import time:       245 |       6390 |               numpy.compat
import time:        27 |       6417 |             numpy.compat._inspect
import time:       437 |      15505 |           numpy.core.overrides
import time:      3372 |      18876 |         numpy.core.multiarray
import time:       250 |        250 |         numpy.core.umath
import time:       492 |        492 |           numbers
import time:       211 |        211 |           numpy.core._string_helpers
import time:       198 |        198 |             numpy.core._dtype
import time:       399 |        597 |           numpy.core._type_aliases
import time:       378 |       1676 |         numpy.core.numerictypes
import time:       570 |        570 |             numpy.core._asarray
import time:       355 |        355 |                 numpy.core._exceptions
import time:       264 |        619 |               numpy.core._methods
import time:      3232 |       3851 |             numpy.core.fromnumeric
import time:       697 |       5117 |           numpy.core.shape_base
import time:       261 |        261 |           numpy.core._ufunc_config
import time:       997 |        997 |           numpy.core.arrayprint
import time:      2027 |       8400 |         numpy.core.numeric
import time:      3157 |       3157 |         numpy.core.defchararray
import time:       432 |        432 |         numpy.core.records
import time:       231 |        231 |         numpy.core.memmap
import time:       519 |        519 |         numpy.core.function_base
import time:       221 |        221 |         numpy.core.machar
import time:       309 |        309 |         numpy.core.getlimits
import time:       502 |        502 |         numpy.core.einsumfunc
import time:       345 |        345 |           numpy.core._multiarray_tests
import time:      1285 |       1630 |         numpy.core._add_newdocs
import time:       446 |        446 |         numpy.core._add_newdocs_scalars
import time:       166 |        166 |         numpy.core._dtype_ctypes
import time:        92 |         92 |             _ast
import time:       465 |        557 |           ast
import time:      2167 |       2167 |           platform
import time:       570 |        570 |             _ctypes
import time:       307 |        307 |             ctypes._endian
import time:       887 |       1764 |           ctypes
import time:       906 |       5392 |         numpy.core._internal
import time:       207 |        207 |         numpy._pytesttester
import time:       693 |      43099 |       numpy.core
import time:       377 |        377 |         numpy.lib.mixins
import time:       388 |        388 |             numpy.lib.ufunclike
import time:       911 |       1298 |           numpy.lib.type_check
import time:       773 |       2071 |         numpy.lib.scimath
import time:      1039 |       1039 |                   numpy.lib.twodim_base
import time:       261 |        261 |                   numpy.linalg.lapack_lite
import time:       304 |        304 |                   numpy.linalg._umath_linalg
import time:      1901 |       3504 |                 numpy.linalg.linalg
import time:       178 |       3681 |               numpy.linalg
import time:       404 |       4085 |             numpy.matrixlib.defmatrix
import time:       170 |       4254 |           numpy.matrixlib
import time:       519 |        519 |             numpy.lib.histograms
import time:      3260 |       3779 |           numpy.lib.function_base
import time:       471 |        471 |           numpy.lib.stride_tricks
import time:       681 |       9184 |         numpy.lib.index_tricks
import time:      1150 |       1150 |         numpy.lib.nanfunctions
import time:      1282 |       1282 |         numpy.lib.shape_base
import time:      1231 |       1231 |         numpy.lib.polynomial
import time:       813 |        813 |         numpy.lib.utils
import time:       789 |        789 |         numpy.lib.arraysetops
import time:       521 |        521 |           weakref
import time:       260 |        260 |           numpy.lib.format
import time:       383 |        383 |               zlib
import time:       254 |        254 |                 _compression
import time:       225 |        225 |                 _bz2
import time:       307 |        785 |               bz2
import time:       328 |        328 |                 _lzma
import time:       292 |        620 |               lzma
import time:        66 |         66 |               pwd
import time:       202 |        202 |               grp
import time:       906 |       2959 |             shutil
import time:       286 |       3244 |           numpy.lib._datasource
import time:       459 |        459 |           numpy.lib._iotools
import time:      1167 |       5649 |         numpy.lib.npyio
import time:       211 |        211 |         numpy.lib.arrayterator
import time:       350 |        350 |         numpy.lib.arraypad
import time:       189 |        189 |         numpy.lib._version
import time:       485 |      23775 |       numpy.lib
import time:       172 |        172 |           numpy.fft._pocketfft_internal
import time:      1498 |       1669 |         numpy.fft._pocketfft
import time:       354 |        354 |         numpy.fft.helper
import time:       220 |       2243 |       numpy.fft
import time:       331 |        331 |           numpy.polynomial.polyutils
import time:       462 |        462 |           numpy.polynomial._polybase
import time:       462 |       1254 |         numpy.polynomial.polynomial
import time:       407 |        407 |         numpy.polynomial.chebyshev
import time:       326 |        326 |         numpy.polynomial.legendre
import time:       349 |        349 |         numpy.polynomial.hermite
import time:       360 |        360 |         numpy.polynomial.hermite_e
import time:       393 |        393 |         numpy.polynomial.laguerre
import time:       268 |       3353 |       numpy.polynomial
import time:       145 |        145 |                 backports_abc
import time:       671 |        815 |               numpy.random._common
import time:       257 |        257 |                   binascii
import time:       298 |        555 |                 base64
import time:       811 |        811 |                   _hashlib
import time:       229 |        229 |                     _blake2
import time:       236 |        236 |                     _sha3
import time:       365 |        830 |                   hashlib
import time:       286 |       1926 |                 hmac
import time:       189 |        189 |                     _bisect
import time:       269 |        458 |                   bisect
import time:       173 |        173 |                   _sha512
import time:       180 |        180 |                   _random
import time:       475 |       1285 |                 random
import time:       216 |       3980 |               secrets
import time:       677 |       5472 |             numpy.random.bit_generator
import time:       267 |        267 |             numpy.random._bounded_integers
import time:       243 |        243 |             numpy.random._mt19937
import time:      1059 |       7039 |           numpy.random.mtrand
import time:       265 |        265 |           numpy.random._philox
import time:       239 |        239 |           numpy.random._pcg64
import time:       218 |        218 |           numpy.random._sfc64
import time:       782 |        782 |           numpy.random._generator
import time:       293 |       8832 |         numpy.random._pickle
import time:       259 |       9091 |       numpy.random
import time:       497 |        497 |       numpy.ctypeslib
import time:       214 |        214 |                 _opcode
import time:       863 |       1076 |               opcode
import time:       506 |       1582 |             dis
import time:       201 |        201 |                 token
import time:      1789 |       1990 |               tokenize
import time:       220 |       2209 |             linecache
import time:      1864 |       5654 |           inspect
import time:      3039 |       8692 |         numpy.ma.core
import time:      1192 |       1192 |         numpy.ma.extras
import time:       280 |      10163 |       numpy.ma
import time:      2181 |      95085 |     numpy
import time:       764 |        764 |     iris.exceptions
import time:       863 |      96902 |   iris._constraints
import time:       561 |        561 |   iris._deprecation
import time:      2062 |       2062 |     configparser
import time:       422 |        422 |       traceback
import time:        62 |         62 |         _string
import time:       724 |        785 |       string
import time:        47 |         47 |       atexit
import time:      2057 |       3310 |     logging
import time:      1432 |       6803 |   iris.config
import time:      2625 |       2625 |   iris.io
import time:       306 |        306 |   iris_sample_data
import time:        56 |         56 |   iris.site_config
import time:       901 |     112356 | iris

@bjlittle
Copy link
Member Author

bjlittle commented Jun 16, 2021

I've been grappling with asv to get some import metrics, and here's the tale of the tape... I'm not entirely sure what asv is doing under the hood (which is somewhat frustrating) but the measures are relative and therefore comparable and so generally relevant here.

The commit SHA's being compared here are from this branch, and compare iris before this PR e0f9792, and with this PR 45a3fbd.

asv-compare

@bjlittle
Copy link
Member Author

bjlittle commented Jun 16, 2021

Regardless of what we do, we need to import what we need to import - there is no magic here.

So at the end of the day there is an import tax to be levied. However we can (to an extent) take explicit control how and when and how much at a time is paid.

There is an obvious regression for iris.plot and iris.quickplot, and I can dig further on that... but for me that then starts to change the intent of this PR.

Overall, I'm pretty happy with this PR "as-is" and I believe it's bankable as it stands - however, I'm going to add an import test to ensure that all iris modules are importable (this test will hopefully catch cyclic imports that get introduced unknowingly)

So I suggest there is a follow-up PR that addresses optimising the imports, and that might result in some restructuring. This PR has certainly make a decent in-road to addressing some ancient tech debt, but I do feel it's open a bit of a can of worms.

Nevertheless, I'm kinda comfortble with what I'm proposing in this PR, now that there are some asv metrics to provide a bit of clarity on import behaviour (which we've never had before).

@trexfeathers
Copy link
Contributor

There is an obvious regression for iris.plot and iris.quickplot, and I can dig further on that... but for me that then starts to change the intent of this PR.

It's more than counteracted by the other improvements, so it can certainly wait for another time.

@bjlittle
Copy link
Member Author

bjlittle commented Jun 16, 2021

It's more than counteracted by the other improvements, so it can certainly wait for another time.

I think so... naively I'm kinda liking it to spreading your toast with butter, it's the same amount of butter but you can choose where you spread it.

Anyways, somehow it "generally" seems like a win, behaviour wise.

Otherwise, I'm a big fan of what isort is offering 👍

@bjlittle
Copy link
Member Author

bjlittle commented Jun 18, 2021

Okay, I revisited asv again... and after some grappling to find the right metrics pattern, we now have much better, stable and representative suite of results, which also align with the python -X importtime -c "import iris" observations... so that gives me more confidence that we've now got reasonable timing asv metrics for the imports:

asv

Note that, these results are based on the following asv benchmark parameters:

  • warmup_time = 0
  • number = 1
  • repeat = 10

Thanks for the awesome asv tech support from @trexfeathers 🍻

@trexfeathers trexfeathers merged commit 15bbcc5 into SciTools:master Jun 18, 2021
@bjlittle
Copy link
Member Author

Awesome, thanks @trexfeathers 🍻🤩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants