Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crc check raises error when the issue is permissions #21

Closed
fcollman opened this issue Aug 27, 2020 · 3 comments
Closed

crc check raises error when the issue is permissions #21

fcollman opened this issue Aug 27, 2020 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@fcollman
Copy link

I received this error in the lastest version of cloud-volume. When i specified use_https=True, it went away. I suspect that the crc checking is failing when a permissions failure is at fault. The error message is not helpful as it behaves now.

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/celery/app/autoretry.py", line 35, in run
    return task._orig_run(*args, **kwargs)
  File "/app/materializationengine/workflows/live_materialization.py", line 321, in get_cloudvolume_supervoxel_ids
    cv = cloudvolume.CloudVolume(segmentation_source, mip=0)
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/cloudvolume.py", line 204, in __new__
    return REGISTERED_PLUGINS[path.format](**kwargs)
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/datasource/graphene/__init__.py", line 69, in create_graphene
    mesh = GrapheneMeshSource(meta, cache, config)
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/datasource/graphene/mesh/__init__.py", line 12, in __new__
    mesh_meta = GrapheneMeshMetadata(meta, cache)
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/datasource/precomputed/mesh/metadata.py", line 17, in __init__
    self.info = self.fetch_info()
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/datasource/precomputed/mesh/metadata.py", line 71, in fetch_info
    info = self.cache.download_json(self.meta.join(self.mesh_path, 'info'))
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/cacheservice.py", line 354, in download_json
    res = self.download( [ path ], compress=compress, progress=False )
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/cacheservice.py", line 477, in download
    raise frag['error']
  File "/usr/local/lib/python3.7/site-packages/cloudfiles/cloudfiles.py", line 174, in download
    check_md5(path, content, server_hash)
  File "/usr/local/lib/python3.7/site-packages/cloudfiles/cloudfiles.py", line 145, in check_md5
    path, server_hash, computed_md5
cloudfiles.exceptions.MD5IntegrityError: graphene_meshes_minnie3_v1/info failed its md5 check. server md5: Y computed md5: Y+GFY7Q7Ph2Pwy6cnb6dVQ==

@william-silversmith william-silversmith self-assigned this Aug 27, 2020
@william-silversmith william-silversmith added the bug Something isn't working label Aug 27, 2020
@william-silversmith
Copy link
Contributor

Hi Forrest,

I ran into this problem myself. The issue is that I didn't parse the md5 correctly in google-cloud-python when I submitted my patch.

googleapis/python-storage#258

You can try switching to an earlier version of CV until the patch comes out. I might release a version of CV that allows you to disable md5 checks depending on how long this will take to fix.

@fcollman
Copy link
Author

ah thanks!

@william-silversmith
Copy link
Contributor

The md5 logic was reactivated a few days ago in the last release. I'm going to close this, but please reopen if you run into problems!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants