'ImportError: No module named pwd' while trying to instantiate a pubsub client

[zenlambda]

We're getting this error when we try to instantiate a PubSub client in our staging environment. This is strange because this code has worked fine for months until today.

Our staging application id is roobeta001.appspot.com.

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/main.py", line 37, in <module>
    excludes=['/login/'])
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/./server/middleware/event_middleware.py", line 226, in create
    sink = get_sink()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/./server/middleware/event_middleware.py", line 219, in get_sink
    return PubSubSink()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/./server/middleware/event_middleware.py", line 108, in __init__
    self.topic = pubsub.Client().topic(topic_name)
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/gcloud/client.py", line 186, in __init__
    Client.__init__(self, credentials=credentials, http=http)
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/gcloud/client.py", line 122, in __init__
    credentials = get_credentials()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/gcloud/credentials.py", line 82, in get_credentials
    return client.GoogleCredentials.get_application_default()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1288, in get_application_default
    return GoogleCredentials._get_implicit_credentials()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1273, in _get_implicit_credentials
    credentials = checker()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1226, in _implicit_credentials_from_files
    credentials_filename = _get_well_known_file()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1392, in _get_well_known_file
    default_config_dir = os.path.join(os.path.expanduser('~'),
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/posixpath.py", line 268, in expanduser
    import pwd
ImportError: No module named pwd

[zenlambda]

We also have other application ids but this issue only occurs on roobeta001. Is there a change to the runtime being rolled out?

[daspecster]

Thanks for reporting @zenlambda!

The issue is due to Appengine not supporting python system modules.
This should be fixed soon though, but make sure you have gcloud-python 0.17.0 or higher installed.

See: #1893 (comment)

[theacodes]

The fix for expanduser should be out in the next release of GAE. In the meantime, you can try to monkeypatch os.path.expanduser in appengine_config.py:

import os.path

def patched_expanduser(path):
    return path

os.path.expanduser = patched_expanduser

[zenlambda]

@daspecster Thanks for the quick response and explanation.

@jonparrott Thanks for the workaround. I've tested it in our environment and it seems to work.

[taohea]

Excuse me, I made a mistake when I visited HDDS with kerberos. I said that I didn't have a PWD module, and I saw many solutions for you, but they were unsuccessful. I have wasted two days, no progress, please see the help! Thank you!
**
Traceback (most recent call last):
File "E:/PycharmProjects/TensorFlow/TestEnd/ImpalaTest.py", line 6, in
from krbcontext import krbcontext
File "D:\Anaconda3\envs\tensorflow\lib\site-packages\krbcontext_init_.py", line 6, in
from .context import krbcontext, krbContext # noqa
File "D:\Anaconda3\envs\tensorflow\lib\site-packages\krbcontext\context.py", line 21, in
import pwd
ModuleNotFoundError: No module named 'pwd'
**