Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"can't use a string pattern on a bytes-like object" on python 3.4 #10

Closed
habnabit opened this issue Sep 11, 2014 · 2 comments
Closed

"can't use a string pattern on a bytes-like object" on python 3.4 #10

habnabit opened this issue Sep 11, 2014 · 2 comments

Comments

@habnabit
Copy link

Running pex on python 3.4 produces the following error, without further details:

pex: Unknown exception encountered: can't use a string pattern on a bytes-like object

Upon hitting C-c, pex emits the following traceback:

Traceback (most recent call last):
  File "ve/bin/pex", line 9, in <module>
    load_entry_point('pex==0.8.0-rc0', 'console_scripts', 'pex')()
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/bin/pex.py", line 308, in main
    pex_builder = build_pex(args, options)
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/bin/pex.py", line 279, in build_pex
    cache_ttl=options.cache_ttl)
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/resolver.py", line 215, in resolve
    existing=distribution_set.get(requirement.key))
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/resolver.py", line 101, in packages_from_requirement_cached
    return packages_from_requirement(iterator, requirement, *args, **kw)
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/resolver.py", line 69, in packages_from_requirement
    return [package for package in existing
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/resolver.py", line 69, in <listcomp>
    return [package for package in existing
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/iterator.py", line 57, in iter
    for package in self._sort(self._iter_unordered(req, follow_links)):
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/iterator.py", line 53, in _sort
    return sorted(package_list, key=key, reverse=True)
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/iterator.py", line 46, in _iter_unordered
    crawled_url_iterator = self._crawler.crawl(url_iterator, follow_links=follow_links)
  File "/home/vagrant/ve/lib/python3.4/site-packages/pex/crawler.py", line 132, in crawl
    queue.join()
  File "/home/vagrant/.local/lib/python3.4/queue.py", line 86, in join
    self.all_tasks_done.wait()
  File "/home/vagrant/.local/lib/python3.4/threading.py", line 290, in wait
    waiter.acquire()
KeyboardInterrupt
@habnabit
Copy link
Author

fwiw this diff fixes it for me on python 3.4, but is probably wrong on python 2.x:

--- a/pex/crawler.py
+++ b/pex/crawler.py
@@ -71,7 +71,7 @@ class Crawler(object):
   @classmethod
   def crawl_remote(cls, context, link):
     try:
-      content = context.read(link)
+      content = context.read(link).decode()
     # except context.Error as e:
     except Exception as e:
       TRACER.log('Failed to read %s: %s' % (link.url, e), V=1)

@Yasumoto
Copy link
Contributor

Hm.. could you catch the exception that py3 throws and then try the decode() then? That (in theory) would give us working py2 + py3, right?

wickman added a commit to pantsbuild/pants that referenced this issue Dec 4, 2014
SSIA.  The user facing parts of this change:
  - package resolution happens via requests now -- this is way more reliable
    and results in fewer Untranslateable exceptions
  - PEX_VERBOSE now controls all pex verbosity, including better messages around Untranslateable (yay!)
  - fixes a major pex regression with namespace packages introduced somewhere in 0.5.x

The pex 0.7.0 -> 0.8.0 changelog:

* *API change*: Decouple translation from package iteration.  This removes
  the Obtainer construct entirely, which likely means if you're using PEX as
  a library, you will need to change your code if you were doing anything
  nontrivial.  This adds a couple new options to ``resolve`` but simplifies
  the story around how to cache packages.
  [RB #785](https://rbcommons.com/s/twitter/r/785/)
* Refactor http handling in pex to allow for alternate http implementations.  Adds support
  for [requests](https://github.com/kennethreitz/requests),
  improving both performance and security.   For more information, read the commit notes at
  [91c7f32](pex-tool/pex@91c7f32).
  [RB #778](https://rbcommons.com/s/twitter/r/778/)
* Improvements to API documentation throughout.
* Renamed ``Tracer`` to ``TraceLogger`` to prevent nondeterministic isort ordering.
* Refactor tox.ini to increase the number of environment combinations and improve coverage.
* Adds HTTP retry support for the RequestsContext.
  [RB #1303](https://rbcommons.com/s/twitter/r/1303/)
* Make pex --version correct.
  [Issue #19](pex-tool/pex#19)
* Bug fix: Fix over-aggressive sys.modules scrubbing for namespace packages.  Under
  certain circumstances, namespace packages in site-packages could conflict with packages
  within a PEX, causing them to fail importing.
  [RB #1378](https://rbcommons.com/s/twitter/r/1378/)
* Bug fix: Replace uses of ``os.unsetenv(...)`` with ``del os.environ[...]``
  [Pull Request #11](pex-tool/pex#11)
* Bug fix: Scrub sys.path and sys.modules based upon both supplied path and
  realpath of files and directories.  Newer versions of virtualenv on Linux symlink site-packages
  which caused those packages to not be removed from sys.path correctly.
  [Issue #21](pex-tool/pex#21)
* Bug fix: The pex -s option was not correctly pulling in transitive dependencies.
  [Issue #22](pex-tool/pex#22)
* Bug fix: Adds ``content`` method to HTTP contexts that does HTML content decoding, fixing
  an encoding issue only experienced when using Python 3.
  [Issue #10](pex-tool/pex#10)

Testing Done:
build-support/bin/ci.sh

Reviewed at https://rbcommons.com/s/twitter/r/1421/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants