Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log during pip install #867

Conversation

j0yu
Copy link
Contributor

@j0yu j0yu commented Mar 22, 2020

Rather than not showing installed rez pip packages because one of the pip packages failed during install, change to log as each rez pip package is installed.

For example:

  1. rez pip --install sphinx
    22:24:27 INFO     Trying to use pip from python package
    22:24:28 INFO     Found pip-19.3.1 inside /root/packages/python/3.7.5/package.py. Will use it with /usr/local/bin/python3
    22:24:28 INFO     Installing 'sphinx' with pip taken from '/usr/local/bin/python3'
    Collecting sphinx
    Downloading https://files.pythonhosted.org/packages/16/c1/1c8016475cb612a64ce613c6b61c16224fe6dcfd9527792016ce611c231c/Sphinx-2.4.4-py3-none-any.whl (2.7MB)
        |████████████████████████████████| 2.7MB 2.5MB/s 
    Collecting sphinxcontrib-htmlhelp (from sphinx)
    Downloading https://files.pythonhosted.org/packages/36/62/8222554b29b3acde8420128d6d3999c5904d40922ef4b6ccb370e2be7421/sphinxcontrib_htmlhelp-1.0.3-py2.py3-none-any.whl (96kB)
        |████████████████████████████████| 102kB 5.4MB/s 
    Collecting ....
    .... certifi>=2017.4.17 (from requests>=2.5.0->sphinx)
    Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB)
        |████████████████████████████████| 163kB 4.8MB/s 
    Installing collected packages: sphinxcontrib-htmlhelp, imagesize, docutils, sphinxcontrib-devhelp, sphinxcontrib-serializinghtml, snowballstemmer, alabaster, pytz, babel, setuptools, MarkupSafe, Jinja2, sphinxcontrib-qthelp, six, pyparsing, packaging, Pygments, sphinxcontrib-applehelp, sphinxcontrib-jsmath, idna, chardet, urllib3, certifi, requests, sphinx
    Successfully installed Jinja2-2.11.1 MarkupSafe-1.1.1 Pygments-2.6.1 alabaster-0.7.12 babel-2.8.0 certifi-2019.11.28 chardet-3.0.4 docutils-0.16 idna-2.9 imagesize-1.2.0 packaging-20.3 pyparsing-2.4.6 pytz-2019.3 requests-2.23.0 setuptools-46.1.1 six-1.14.0 snowballstemmer-2.0.0 sphinx-2.4.4 sphinxcontrib-applehelp-1.0.2 sphinxcontrib-devhelp-1.0.2 sphinxcontrib-htmlhelp-1.0.3 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.3 sphinxcontrib-serializinghtml-1.1.4 urllib3-1.25.8
    WARNING: You are using pip version 19.1.1, however version 20.0.2 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    22:24:38 INFO     Installed [Babel-2.8.0] /root/packages/Babel/2.8.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [Jinja2-2.11.1] /root/packages/Jinja2/2.11.1/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [MarkupSafe-1.1.1] /root/packages/MarkupSafe/1.1.1/package.py (d9e9d80193dcd9578844ec4c2c22c9366ef0b88a)
    22:24:38 INFO     Installed [Pygments-2.6.1] /root/packages/Pygments/2.6.1/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [Sphinx-2.4.4] /root/packages/Sphinx/2.4.4/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [alabaster-0.7.12] /root/packages/alabaster/0.7.12/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [certifi-2019.11.28] /root/packages/certifi/2019.11.28/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [chardet-3.0.4] /root/packages/chardet/3.0.4/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [docutils-0.16] /root/packages/docutils/0.16/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [idna-2.9] /root/packages/idna/2.9/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [imagesize-1.2.0] /root/packages/imagesize/1.2.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [packaging-20.3] /root/packages/packaging/20.3/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [pyparsing-2.4.6] /root/packages/pyparsing/2.4.6/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [pytz-2019.3] /root/packages/pytz/2019.3/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:38 INFO     Installed [requests-2.23.0] /root/packages/requests/2.23.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [setuptools-46.1.1] /root/packages/setuptools/46.1.1/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [six-1.14.0] /root/packages/six/1.14.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [snowballstemmer-2.0.0] /root/packages/snowballstemmer/2.0.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [sphinxcontrib_applehelp-1.0.2] /root/packages/sphinxcontrib_applehelp/1.0.2/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [sphinxcontrib_devhelp-1.0.2] /root/packages/sphinxcontrib_devhelp/1.0.2/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [sphinxcontrib_htmlhelp-1.0.3] /root/packages/sphinxcontrib_htmlhelp/1.0.3/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [sphinxcontrib_jsmath-1.0.1] /root/packages/sphinxcontrib_jsmath/1.0.1/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [sphinxcontrib_qthelp-1.0.3] /root/packages/sphinxcontrib_qthelp/1.0.3/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [sphinxcontrib_serializinghtml-1.1.4] /root/packages/sphinxcontrib_serializinghtml/1.1.4/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     Installed [urllib3-1.25.8] /root/packages/urllib3/1.25.8/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:24:39 INFO     25 packages were installed.
    

    ...then...

  2. rez pip --install pytest-sphinx
    22:30:46 INFO     Trying to use pip from python package
    22:30:46 INFO     Found pip-19.3.1 inside /root/packages/python/3.7.5/package.py. Will use it with /usr/local/bin/python3
    22:30:46 INFO     Installing 'pytest-sphinx' with pip taken from '/usr/local/bin/python3'
    Collecting pytest-sphinx
    Downloading https://files.pythonhosted.org/packages/9d/f9/69caa3584c5b8fc5b9cce04dc9d3d538c50ae1b0020f5a6538d505011288/pytest-sphinx-0.2.2.tar.gz
    Installing build dependencies ... done
    Getting requirements to build wheel ... done
        Preparing wheel metadata ... done
    Collecting pytest>=3.1.1 (from pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/c7/e2/c19c667f42f72716a7d03e8dd4d6f63f47d39feadd44cc1ee7ca3089862c/pytest-5.4.1-py3-none-any.whl (246kB)
        |████████████████████████████████| 256kB 3.0MB/s 
    Collecting pluggy<1.0,>=0.12 (from pytest>=3.1.1->pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/a0/28/85c7aa31b80d150b772fbe4a229487bc6644da9ccb7e427dd8cc60cb8a62/pluggy-0.13.1-py2.py3-none-any.whl
    Collecting wcwidth (from pytest>=3.1.1->pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/58/b4/4850a0ccc6f567cc0ebe7060d20ffd4258b8210efadc259da62dc6ed9c65/wcwidth-0.1.8-py2.py3-none-any.whl
    Collecting py>=1.5.0 (from pytest>=3.1.1->pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/99/8d/21e1767c009211a62a8e3067280bfce76e89c9f876180308515942304d2d/py-1.8.1-py2.py3-none-any.whl (83kB)
        |████████████████████████████████| 92kB 5.7MB/s 
    Collecting more-itertools>=4.0.0 (from pytest>=3.1.1->pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/72/96/4297306cc270eef1e3461da034a3bebe7c84eff052326b130824e98fc3fb/more_itertools-8.2.0-py3-none-any.whl (43kB)
        |████████████████████████████████| 51kB 7.0MB/s 
    Collecting attrs>=17.4.0 (from pytest>=3.1.1->pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl
    Collecting importlib-metadata>=0.12; python_version < "3.8" (from pytest>=3.1.1->pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/8b/03/a00d504808808912751e64ccf414be53c29cad620e3de2421135fcae3025/importlib_metadata-1.5.0-py2.py3-none-any.whl
    Collecting packaging (from pytest>=3.1.1->pytest-sphinx)
    Using cached https://files.pythonhosted.org/packages/62/0a/34641d2bf5c917c96db0ded85ae4da25b6cd922d6b794648d4e7e07c88e5/packaging-20.3-py2.py3-none-any.whl
    Collecting zipp>=0.5 (from importlib-metadata>=0.12; python_version < "3.8"->pytest>=3.1.1->pytest-sphinx)
    Downloading https://files.pythonhosted.org/packages/b2/34/bfcb43cc0ba81f527bc4f40ef41ba2ff4080e047acb0586b56b3d017ace4/zipp-3.1.0-py3-none-any.whl
    Collecting pyparsing>=2.0.2 (from packaging->pytest>=3.1.1->pytest-sphinx)
    Using cached https://files.pythonhosted.org/packages/5d/bc/1e58593167fade7b544bfe9502a26dc860940a79ab306e651e7f13be68c2/pyparsing-2.4.6-py2.py3-none-any.whl
    Collecting six (from packaging->pytest>=3.1.1->pytest-sphinx)
    Using cached https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
    Building wheels for collected packages: pytest-sphinx
    Building wheel for pytest-sphinx (PEP 517) ... done
    Stored in directory: /root/.cache/pip/wheels/bd/e0/9e/2aa586098164b15411fd6db48c1489e9e8495d4609cdbdade3
    Successfully built pytest-sphinx
    Installing collected packages: zipp, importlib-metadata, pluggy, wcwidth, py, more-itertools, attrs, pyparsing, six, packaging, pytest, pytest-sphinx
    Successfully installed attrs-19.3.0 importlib-metadata-1.5.0 more-itertools-8.2.0 packaging-20.3 pluggy-0.13.1 py-1.8.1 pyparsing-2.4.6 pytest-5.4.1 pytest-sphinx-0.2.2 six-1.14.0 wcwidth-0.1.8 zipp-3.1.0
    WARNING: You are using pip version 19.1.1, however version 20.0.2 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    22:30:50 INFO     Installed [attrs-19.3.0] /root/packages/attrs/19.3.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:30:50 INFO     Installed [importlib_metadata-1.5.0] /root/packages/importlib_metadata/1.5.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:30:50 INFO     Installed [more_itertools-8.2.0] /root/packages/more_itertools/8.2.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:30:50 WARNING  Skipping installation: Package variant already exists: /root/packages/packaging/20.3/package.py[0]
    22:30:50 INFO     Installed [pluggy-0.13.1] /root/packages/pluggy/0.13.1/package.py (0195e0b9d4f737afb12e188f1abf4b9e01abb774)
    22:30:50 INFO     Installed [py-1.8.1] /root/packages/py/1.8.1/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:30:50 WARNING  Skipping installation: Package variant already exists: /root/packages/pyparsing/2.4.6/package.py[0]
    22:30:50 INFO     Installed [pytest-5.4.1] /root/packages/pytest/5.4.1/package.py (0195e0b9d4f737afb12e188f1abf4b9e01abb774)
    22:30:50 INFO     Installed [pytest_sphinx-0.2.2] /root/packages/pytest_sphinx/0.2.2/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:30:50 WARNING  Skipping installation: Package variant already exists: /root/packages/six/1.14.0/package.py[0]
    22:30:51 INFO     Installed [wcwidth-0.1.8] /root/packages/wcwidth/0.1.8/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:30:51 INFO     Installed [zipp-3.1.0] /root/packages/zipp/3.1.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
    22:30:51 INFO     9 packages were installed.
    22:30:51 WARNING  3 packages were already installed.
    

Changed

  • Moved printing installed variants from rez.cli.pip.command() to info and
    warning logging in rez.pip.pip_install_package().
  • Compacted per variant install message
  • Log debug if variant already exists. PackageMaker already warns about
    existing package variants, log debug for exact variant sub-path.
  • During rez-pip, force logging.INFO for top-level rez logger to prevent
    other loggers from printing debug unless verbose

Fixed

  • Original debug_package_release not making rez-pip produce debug
    messages when True and no rez-pip --verbose

@nerdvegas nerdvegas merged commit bccd919 into AcademySoftwareFoundation:master Mar 31, 2020
@j0yu j0yu deleted the feature-log-pip-while-installing branch March 31, 2020 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants