API 403 errors

[ahamelers]

Bug description:

API user initially reported receiving 403 errors while attempting to download an unpublished dataset. Ryan was able to verify this (see steps to reproduce below). The user is now reporting that they are unable to download individual files or to update datasets as well.

We further investigated the Dryad issue and we concluded that integration in
essence no longer works. We are still able to create datasets and delete
documents but update and all download options no longer work. The issues
started occurring as far as March and we think users were not reporting it
since individual file download worked. Individual file download no longer
works. For a while file update worked, but it no longer works either. We
were getting various errors from 500 (Internal server error), 403 (requested
resource is forbidden) to 405 (Method Not Allowed) reporting files are too
big. Since we are getting inconsistent errors for the same calls we are
really not sure what is going on their end. They need to investigate on
their end.

The 405 error is fine but the others shouldn't be happening.

Steps to reproduce:

1. This works: curl -i -X GET https://datadryad.org/api/v2/datasets/doi%3A10.5061%2Fdryad.b2rbnzspw -H "Authorization: Bearer <some_token>"
2. If you add /download, it doesn’t work, gives a 403. (The dataset is not yet published.)

Expected behavior:

API users who create datasets should be able to download and to update those datasets

[ahamelers]

API downloads work locally, but not on sandbox or prod. I've tried pushing some rack_attack changes with no effect. As far as I can see looking at our permissions setup, any user who can see dataset metadata including a download link should also be able to download that dataset (and the files thereof).

I think the AWS WAF settings are now the most likely suspect here.

[ahamelers]

It was the AWS WAF settings.