Is it possible to provide sync permission to the AWS bucket?

[knight42]

We are hosting a mirror and find it somewhat difficult to sync the files. Could admin allow others to sync the files directly from AWS S3 server, i.e. grant the others list permission?

[zhsj]

We need to setup the mirror since aws s3 is blocked in China. And if we can use the aws s3 api to do the sync operation, it will be very convenient.

[alexcrichton]

Right now the story for mirrors is unfortunately not fantastic, although I would expect any form of sync'ing API to happen via the HTTP server as a "source of truth" as it would allow us to possible switch away from AWS as a backend in the future.

Along those lines I don't think that we'll give direct access to the S3 bucket itself, but instead for now I'm gonna close this in favor of #67 which is tracking setting up a mirror.

[zhsj]

@alexcrichton , I see the #67 is a long-term issue. And it seems inactive for a long time.

We want to set up the mirror immediately because S3 can't be accessed from some country like China.
And we received a lot of requests to provide such a mirror.

Right now the crates are hosted on S3, and it can't be synced using tools like rsync. But it can be synced using S3 api as long as the crates' bucket open the list permission to public. Otherwise we need to write a tool to manually diff what updated in crates, and then download it.

[alexcrichton]

Yeah it's unfortunately not the best of situations, but that's the most that we'd want to commit to at this time. I wouldn't want our hands tied to S3, and the JSON API of crates.io should be pretty simple to write a tool for I believe to do the syncing