Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Offline downloads (called by atlas_occurrences) fail for some atlases #156

Closed
mjwestgate opened this issue Sep 14, 2022 · 4 comments
Closed

Offline downloads (called by atlas_occurrences) fail for some atlases #156

mjwestgate opened this issue Sep 14, 2022 · 4 comments

Comments

@mjwestgate
Copy link
Collaborator

mjwestgate commented Sep 14, 2022 •
edited
Loading

Affected atlases:

Using Estonia as an example; this atlas appears to have an offline downloads API at https://elurikkus.ee/biocache-service/occurrences/offline/download. While using wait_for_download() (within atlas_occurrences), I got the following status URL:

https://elurikkus.ee/biocache-service/occurrences/offline/status/5fe609b1-370f-3f3f-8482-8e46c6d94551-1663130411098

This resolves to:

{"status":"finished","downloadUrl":"http://biocache.ala.org.au/biocache-download/5fe609b1-370f-3f3f-8482-8e46c6d94551/1663130411098/data.zip"}

Clearly this isn't the correct downloadUrl. Manually replacing http://biocache.ala.org.au/ with https://elurikkus.ee/ also fails.

For France, the status queue info returned is:

$message
[1] "Already in queue."

$status
[1] "inQueue"

$queueSize
[1] 1

$statusUrl
[1] "http://biocacheservice:8080/biocache-service/occurrences/offline/status/6a784afe-b089-30b4-ac31-6e64e82dffc5-1663201057954"

$searchUrl
[1] "http://localhost/ala-hub/occurrences/search?&q=*%3A*&fq=%28year%3A%221970%22%29+AND+%28order%3A%22Lagomorpha%22%29"

attr(,"url")
[1] "http://portail.gbif.fr/biocache-service/occurrences/offline/download?fq=%28year%3A%221970%22%29%20AND%20%28order%3A%22Lagomorpha%22%29&fields=taxon_name%2Cyear&qa=none&emailNotify=false&sourceTypeId=2004&reasonTypeId=4&email=ala4r%40ala.org.au&dwcHeaders=true"

Portugal generates a download url that appears valid, but does not resolve:

https://registos.gbif.pt/biocache-download/5fe609b1-370f-3f3f-8482-8e46c6d94551/1663202568148/data.zip

UK sends an email notifying that download has failed, but as with Portugal, I haven't worked out why/how yet.
@mjwestgate mjwestgate added bug Something isn't working and removed bug Something isn't working labels Sep 14, 2022
@mjwestgate mjwestgate changed the title Issues with atlas = "Estonia" Issues with International atlases Sep 15, 2022
@mjwestgate mjwestgate changed the title Issues with International atlases Offline downloads (called by atlas_occurrences) fail for some atlases Oct 27, 2022
@vjrj
Copy link

vjrj commented Oct 28, 2022

cc @keithraven-nbn @rpfigueira @sylmorin-gbif

@vjrj
Copy link

vjrj commented Oct 28, 2022

Probably the same like #165 (comment)

mjwestgate added a commit that referenced this issue Jan 6, 2023
@mjwestgate
update to improve tests & associated performance
251f82c 
- add features for Spain (gbif.es)
- swap France to OpenObs (#165, #156). This is in progress and needs further work
- fix bug in collectory `show_all` functions where NULL returns caused an error
- cleaned tests for international atlases to ensure better performance and more accurate diagnosis of bugs (Sweden and UK remaining)
- update NEWS.md
@mjwestgate
Copy link
Collaborator Author

This is still an issue for Portugal and Estonia; UK and France supported as of version 2.0

@mjwestgate
Copy link
Collaborator Author

Gonna close this one; we can add more specific errors in due course

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vjrj @mjwestgate