-
Notifications
You must be signed in to change notification settings - Fork 137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Alipan Storage backend #651
Conversation
* Add `download_from_alipan` function and `AlipanUploader` class * Add tests for both download and upload * Add docs for configure Alipan Storage Credentials * Add `AliPCS-Py` to setup.py as extra dependence
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your contribution. Posted some minor comments.
* Use accurate `download_file` instead of the general `download` when download a remote file * Raise a `ValueError` when the remote is `None` at uploading * Format code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @PeterDing, just getting back to this now. Left a couple of suggestions and comments. Are there any other changes you'd like to make?
docs/source/how_to_guides/configure_cloud_storage_credentials.md
Outdated
Show resolved
Hide resolved
docs/source/how_to_guides/configure_cloud_storage_credentials.md
Outdated
Show resolved
Hide resolved
Co-authored-by: Saaketh Narayan <saaketh@mosaicml.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm! this will be included in the upcoming release -- thanks for contributing!
@PeterDing It would be most helpful if we could refer users to you if they face issues with Alipan. You are much more well versed with Alipan stuff than our team :) Excited to get this integrated into Streaming! |
Add support to access Alipan Storage.
Description of changes:
Add
download_from_alipan
function andAlipanUploader
classAdd tests for both download and upload
Add docs for configure Alipan Storage Credentials
Add
AliPCS-Py
to setup.py as extra dependenceIssue #, if available:
Merge Checklist:
Put an
x
without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.General
Tests
pre-commit
on my change. (check out thepre-commit
section of prerequisites)