Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support GBIF queries from galah #172

Closed
mjwestgate opened this issue Dec 7, 2022 · 1 comment
Closed

Support GBIF queries from galah #172

mjwestgate opened this issue Dec 7, 2022 · 1 comment
Labels
enhancement New feature or request

Comments

@mjwestgate
Copy link
Collaborator

In a previous issue (#126) we expanded support to all available living atlases. We currently (v 1.5.0) support the GBIF name-matching service for some atlases, but no other services; I suspect this is a strange oversight to most users.

The first set of challenges to upgrading to GBIF are:

  • No GBIF API for field names, assertions etc (rgbif hard codes them)
  • Need for new translators in galah_filter as GBIF doesn't use solr
  • The 'predicates' system for GBIF downloads is analogous to our solr approach (albeit in JSON), but the 'online' GBIF occurrence API doesn't use them, potentially creating a mismatch between atlas_counts (online) and atlas_occurrences (offline)

Other problems may emerge once these are solved. However, the two systems are otherwise closely related, so it is conceptually straightforward to map current galah functions to GBIF.
@mjwestgate mjwestgate added the enhancement New feature or request label Dec 7, 2022
mjwestgate added a commit that referenced this issue Dec 7, 2022
@mjwestgate
Add show_all functions for GBIF (#172)
6613a73 
- web-scrape data for `fields` and `assertions`
- support `collections` and `providers` via API with pagination
- `show_all_datasets` too large to work (~80k), but API supports queries, so `search_all_datasets` could work
mjwestgate added a commit that referenced this issue Dec 7, 2022
@mjwestgate
Simple system for building predicates for the GBIF download system (#172
c8b9726 
)

Takes an input from `galah_filter` and returns a JSON to pass to GBIF
mjwestgate added a commit that referenced this issue Dec 9, 2022
@mjwestgate
Major cleanup of backend code
8c895c6 
- rename all code that directly calls APIs from `atlas_` prefix to `url_` prefix, to avoid confusion with functions that return data
- restructure `galah_config` for cleaner storage, ability to specify atlas names by org name, acronym or region
- add new arguments `username` and `password` to `galah_config` to support gbif integration (#172)
- restructure `url_` functions to no longer use `tryCatch`
- parse out `url_get` from `url_paginate` to make code cleaner and easier to use ahead of new API calls (#163)
mjwestgate added a commit that referenced this issue Dec 22, 2022
@mjwestgate
Support atlas_occurrences when atlas = GBIF (#172)
47699c1 
- upgrade `atlas_occurrences` to check whether `atlas = GBIF` and send correct arguments to correct functions
- split `atlas_occurrences_internal` into Living Atlases implementation `_LA` and `_GBIF`
- update `build_predicates` to match output of `rgbif`
- misc changes to `url_GET`, `url_POST` and `url_download`
- convert `check_queue` to `url_queue` for consistency with related functions
mjwestgate added a commit that referenced this issue Dec 23, 2022
@mjwestgate
support atlas_species when atlas = "GBIF" (#172)
585ebf8 
Unlike ALA, which has a separate API for species downloads (i.e. facets downlods), GBIF works through the occurrences download system. Ergo we can use the same code for `atlas_occurrences` and `atlas_species`, simply by changing `format` from `SIMPLE_CSV` to `SPECIES_LIST`. This commit makes the relevant changes, but doesn't stabilise `occurrences_GBIF` any further, meaning more checks will be needed
mjwestgate added a commit that referenced this issue Jan 4, 2023
@mjwestgate
add tests for common functions when atlas = GBIF (#172)
af2e617 
+ following improvements to GBIF functions:
- limit results from `show_all` collectory functions to n = 20, with a message
- add `search_all` collectory functions
- support `group_by` and range queries for `atlas_counts`
- remove support for `taxize` objects
- add `is_gbif()` function for easy lookup
- fix bugs in `print.galah_config` & `url_paginate`
mjwestgate added a commit that referenced this issue Jan 6, 2023
@mjwestgate
complete integration of atlas_occurrences and atlas_species for G…
3251595 
…BIF (#172)

- add tests for `atlas_species` and `atlas_occurrences`
- provide more useful progress information when verbose = TRUE
- update `build_filter_query` to support `==`, `<=` or `>=` BUT NOT `<`, `>` or `!=` (relevant to `atlas_counts`)
mjwestgate added a commit that referenced this issue Jan 13, 2023
@mjwestgate
Fix bug where GBIF downloads failed because sendNotification was a …
4b9a6b5 
…string, and not logical (#172)
@mjwestgate
Copy link
Collaborator Author

Closing this with release of v1.5.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mjwestgate