Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pgstac v0.4.0b #308

Merged
merged 8 commits into from
Dec 14, 2021
Merged

Pgstac v0.4.0b #308

merged 8 commits into from
Dec 14, 2021

Conversation

bitner
Copy link
Collaborator

@bitner bitner commented Dec 13, 2021

Related Issue(s): #283
This PR should supersede #307 and #305

Description:

  • Upgrades pgstac backend to 0.4.0
  • Enables cql2-json
    • pgstac enables as default (settable in pgstac configuration)
    • can be set per query using "filter-lang" parameter ("cql-json" or "cql2-json")
    • stac-fastapi-pgstac will automatically use cql-json if query, bbox, ids, collections, or datetime parameter are set for backwards compatibility

PR Checklist:

  • Code is formatted and linted (run pre-commit run --all-files)
  • Tests pass (run make test)
  • Documentation has been updated to reflect changes, if applicable, and docs build successfully (run make docs)
  • Changes are added to the CHANGELOG.

@bitner bitner marked this pull request as ready for review December 13, 2021 23:17
@bitner
Copy link
Collaborator Author

bitner commented Dec 13, 2021

@moradology @edkeeble This PR includes the changes from #305 and #307 adding some basic tests to make sure that cql-json and cql2-json are both working. Would be great if you could throw in a review.

Copy link
Collaborator

@moradology moradology left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. And thanks for the tests. LGTM 👍

Copy link
Contributor

@edkeeble edkeeble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks for wrapping this up @bitner!

I had one question around the filter_lang validator and whether we want to transform cql-text to cql-json, as well as a suggestion for avoiding some duplicate code, but it isn't critical. If the validator works as you intended, then it's good with me.

Comment on lines +27 to +43
retval = v
if values.get("query", None) is not None:
retval = "cql-json"
if values.get("collections", None) is not None:
retval = "cql-json"
if values.get("ids", None) is not None:
retval = "cql-json"
if values.get("datetime", None) is not None:
retval = "cql-json"
if values.get("bbox", None) is not None:
retval = "cql-json"
if v == "cql2-json" and retval == "cql-json":
raise ValueError(
"query, collections, ids, datetime, and bbox"
"parameters are not available in cql2-json"
)
return retval
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cql-text and cql-json share the parameters listed here, but if filter_lang is "cql-text", it will be transformed to "cql-json". Is that the intention?

We might be able to condense this and avoid some duplicate code with the following snippet. It does change the logic slightly, though:

  • It will throw if any CQL parameter is provided, even if its value is null. e.g. {"filter-lang": "cql2-json", "query": null} will throw the ValueError. I feel like that is still correct.
  • It will only default to cql-json if filter-lang isn't specified (i.e. it will preserve cql-text)
cql_params = ("query", "collections", "ids", "datetime", "bbox")
if any(cql_param in values for cql_param in cql_params):
    if v == FilterLang.cql2_json:
        raise ValueError(f"{', '.join(invalid_cql2_params[:-1])} and {invalid_cql2_params[-1]} parameters are not available in cql2-json")
    else:
        return v or FilterLang.cql_json  # if `filter-lang` isn't specified and CQL parameters are provided, default to `cql-json`
return v

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PGStac only accepts json, so in the case of a GET request, that request is just modified into CQL json. The ultimate plan (which I will start working on this week, but probably won't finish before the holidays) for stac-fastapi-pgstac will be to add a cql2-json output to pygeofilter and to use pygeofilter to translate cql-text or cql2-text into cql2-json and then send that to pgstac.

@moradology moradology merged commit 256ae3d into stac-utils:master Dec 14, 2021
@vincentsarago
Copy link
Member

Is there anything here that should be added in the changelog @moradology @bitner ?

@moradology
Copy link
Collaborator

Probably not a bad idea to mention the bump and CQL2 support. I'll toss up a PR for that. Thanks for keeping an eye out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants