Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support URLFile in the upload_file function #1985

Merged
merged 4 commits into from
Oct 16, 2024
Merged

Support URLFile in the upload_file function #1985

merged 4 commits into from
Oct 16, 2024

Conversation

aron
Copy link
Contributor

@aron aron commented Oct 11, 2024

We would like predict functions to be able to return a remote URL rather than a local file on disk and have it behave like a file object. And when it is passed to the file uploader it will stream the file from the remote to the destination provided.

class Predictor(BasePredictor):
    def predict(self, **kwargs) -> File:
        return URLFile("https://replicate.delivery/czjl/9MBNrffKcxoqY0iprW66NF8MZaNeH322a27yE0sjFGtKMXLnA/hello.webp")

This PR adds an additional check to the upload_file() handler to call fh.seekable() before attempting to seek. This allows instances of io.IOBase that do not support seek (like URLFile) to be uploaded.

We also add a name attribute to URLFile. This is used by the upload_file function to infer the file extension and mime type.

@aron aron changed the title upload url file Support URLFile in the upload_file function Oct 11, 2024
python/cog/types.py Outdated Show resolved Hide resolved
Copy link
Member

@nickstenning nickstenning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the filename extraction which doesn't really work for URLs in general, this LGTM.

@nickstenning
Copy link
Member

I think we need to limit this to http/https/data URLs for security reasons, as noted in #1987.

This commit adds an additional check to the `upload_file()` handler to
call `fh.seekable()` before attempting to seek. This allows instances
of `io.IOBase` that do not support seek (like URLFile) to be uploaded.
This is used by the `upload_file` function to infer the file extension
and mime type.
@aron aron merged commit 87787f4 into main Oct 16, 2024
20 checks passed
@aron aron deleted the upload-url-file branch October 16, 2024 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants