added xronos to getter functions #150
Conversation
| site,xronos,site | ||
| ,xronos,site_phase | ||
| feature,xronos,feature | ||
| ,xronos,period |
There was a problem hiding this comment.
Wouldn't that fit to c14bazAAR.period, which is defined as
Historico-cultural period of sample context
and usually contains values like "Bronze Age" etc.
| ,xronos,site_phase | ||
| feature,xronos,feature | ||
| ,xronos,period | ||
| ,xronos,typochronological_unit |
There was a problem hiding this comment.
That should fit to c14bazAAR.culture, right?
There was a problem hiding this comment.
btw. the variable definition table is available here: https://github.com/ropensci/c14bazAAR/blob/master/data-raw/variable_definition.csv
| #' @export | ||
| get_xronos <- function(db_url = get_db_url("xronos")) { | ||
|
|
||
| check_connection_to_url(db_url) |
There was a problem hiding this comment.
Does this already download the whole database, because db_url points to the endpoint at https://xronos.ch/api/v1/data/? It takes forever. Probably it's sufficient to check if https://www.xronos.ch is up here. Or maybe the API has a more minimal endpoint to check if it's available.
|
|
||
| check_connection_to_url(db_url) | ||
|
|
||
| resp<-httr::GET(db_url) |
There was a problem hiding this comment.
Here comes a big one: I know you like proper APIs - rightfully so - but for c14bazAAR they are not ideal. Is there a chance we could get a setup like the one for radon again? Every night the server creates a simple .csv flat file with the columns we need?
Maybe I'm completely off and this is already the fastest solution. Happy to be disabused. But I have the feeling the simple interface would be significantly faster, even for the massive amount of data in xronos. Maybe c14bazAAR is too specific of an application to consider this.
There was a problem hiding this comment.
I know it's unfair, but look at that:
> system.time(get_calpal() -> test1)
user system elapsed
0.430 0.027 0.417
> system.time(get_radon() -> test3)
user system elapsed
0.444 0.045 0.887
> system.time(get_xronos() -> test2)
user system elapsed
75.529 0.617 188.656
This version of CalPal has 49834, so more than half of the 92126 dates in xronos.
|
|
||
| xronos_data <- jsonRespParsed %>% tidyr::tibble() %>% | ||
| tidyr::unnest_wider(1) %>% | ||
| tidyr::unnest_wider(measurement) |
There was a problem hiding this comment.
These procedures take a significant amount of time and could be avoided with a simple flat file download.... 😅
|
@nevrome @MartinHinz We now finally provide a flat file with all our data (https://xronos.ch/data.csv; see xronos-ch/xronos.rails#115) – perhaps time to revisit this? |
|
Neat! Would be easy to do this now, I guess. Cool that you decided to include the flat-file version 🥳 |
|
Probably it would be easier to create a new PR for this. This one is a bit dated... |
time to give back ;-)