Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Maps] Do not fetch geo_shape from docvalues #63997

Merged
merged 1 commit into from
Apr 20, 2020

Conversation

nreese
Copy link
Contributor

@nreese nreese commented Apr 20, 2020

geo_points are fetched from doc_values to get a consistently formatted value. The logic is written very generically so that any field that supports doc values will be fetched with doc values.

Elasticsearch has recently merged support for geo_shape docvalues (elastic/elasticsearch#55037). Looks like there is a bug when requesting doc values for geo_shape fields. This is just a quick fix to not request doc values until the bug for geo_shapes gets figured out.

Example data set with geo_shape field.

PUT /example
{
    "mappings": {
        "properties": {
            "location": {
                "type": "geo_shape"
            }
        }
    }
}
POST /example/_doc
{
    "location" : {
        "type" : "linestring",
        "coordinates" : [[-77.03653, 38.897676], [-77.009051, 38.889939]]
    }
}

Working query

GET example/_search
{
  "query": {
    "match_all": {}
  }
}

Requesting docvalue_fields fails.

GET shapes/_search
{
  "query": {
    "match_all": {}
  },
  "docvalue_fields": ["location"]
}
{
  "error" : {
    "root_cause" : [
      {
        "type" : "unsupported_operation_exception",
        "reason" : "scripts and term aggs are not supported by geo_shape doc values"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "example",
        "node" : "Ef8381lmQ82FBAh45eWkZA",
        "reason" : {
          "type" : "unsupported_operation_exception",
          "reason" : "scripts and term aggs are not supported by geo_shape doc values"
        }
      }
    ],
    "caused_by" : {
      "type" : "unsupported_operation_exception",
      "reason" : "scripts and term aggs are not supported by geo_shape doc values",
      "caused_by" : {
        "type" : "unsupported_operation_exception",
        "reason" : "scripts and term aggs are not supported by geo_shape doc values"
      }
    }
  },
  "status" : 500
}

@nreese nreese added [Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.8.0 labels Apr 20, 2020
@nreese nreese requested a review from thomasneirynck April 20, 2020 19:55
@nreese nreese requested a review from a team as a code owner April 20, 2020 19:55
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-gis (Team:Geo)

Copy link
Contributor

@thomasneirynck thomasneirynck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the quick turn-around.

Can you link the ES bug report here too? Ideally, this gets returned from ES as expected.

@nreese
Copy link
Contributor Author

nreese commented Apr 20, 2020

opened elastic/elasticsearch#55487 to track issue on Elasticsearch side

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@nreese nreese merged commit 35dab18 into elastic:master Apr 20, 2020
nreese added a commit to nreese/kibana that referenced this pull request Apr 20, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request Apr 21, 2020
* master: (30 commits)
  Migrate vis_type_table to kibana/new platform  (elastic#63105)
  Enable include/exclude in Terms agg for numeric fields (elastic#59425)
  follow conventions for saved object definitions (elastic#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838)
  Fix page layouts, clean up unused code (elastic#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859)
  [Maps] Do not fetch geo_shape from docvalues (elastic#63997)
  Vega doc changes (elastic#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049)
  Add sub-menus to Resolver node (for 75% zoom) (elastic#63476)
  [FTR] Add test suite metrics tracking/output (elastic#62515)
  [ML] Fixing single metric viewer page padding (elastic#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623)
  [Reporting] Config flag to escape formula CSV values (elastic#63645)
  [Metrics UI] Remove remaining field filtering (elastic#63398)
  [Maps] fix date labels (elastic#63909)
  [TSVB] Use default Kibana palette for split series (elastic#62241)
  Ingest overview page (elastic#63214)
  ...
jloleysens added a commit that referenced this pull request Apr 21, 2020
…t-node-pipelines

* 'master' of github.com:elastic/kibana: (32 commits)
  Move authz lib out of snapshot restore (#63947)
  Migrate vis_type_table to kibana/new platform  (#63105)
  Enable include/exclude in Terms agg for numeric fields (#59425)
  follow conventions for saved object definitions (#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (#63838)
  Fix page layouts, clean up unused code (#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (#63859)
  [Maps] Do not fetch geo_shape from docvalues (#63997)
  Vega doc changes (#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (#60049)
  Add sub-menus to Resolver node (for 75% zoom) (#63476)
  [FTR] Add test suite metrics tracking/output (#62515)
  [ML] Fixing single metric viewer page padding (#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (#63623)
  [Reporting] Config flag to escape formula CSV values (#63645)
  [Metrics UI] Remove remaining field filtering (#63398)
  [Maps] fix date labels (#63909)
  [TSVB] Use default Kibana palette for split series (#62241)
  ...
jloleysens added a commit to jloleysens/kibana that referenced this pull request Apr 21, 2020
…bana into ingest-node-pipelines/privileges

* 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (34 commits)
  Move authz lib out of snapshot restore (elastic#63947)
  Migrate vis_type_table to kibana/new platform  (elastic#63105)
  Enable include/exclude in Terms agg for numeric fields (elastic#59425)
  follow conventions for saved object definitions (elastic#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838)
  Fix page layouts, clean up unused code (elastic#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859)
  [Maps] Do not fetch geo_shape from docvalues (elastic#63997)
  Vega doc changes (elastic#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049)
  Add sub-menus to Resolver node (for 75% zoom) (elastic#63476)
  [FTR] Add test suite metrics tracking/output (elastic#62515)
  [Ingest pipelines] Delete pipeline (elastic#63635)
  [ML] Fixing single metric viewer page padding (elastic#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623)
  [Reporting] Config flag to escape formula CSV values (elastic#63645)
  [Metrics UI] Remove remaining field filtering (elastic#63398)
  [Maps] fix date labels (elastic#63909)
  ...

# Conflicts:
#	x-pack/legacy/plugins/uptime/public/components/monitor/ml/index.ts
#	x-pack/legacy/plugins/uptime/public/components/overview/index.ts
#	x-pack/plugins/ingest_pipelines/public/application/index.tsx
#	x-pack/plugins/ingest_pipelines/server/routes/api/index.ts
#	x-pack/plugins/ingest_pipelines/server/routes/index.ts
jloleysens added a commit to jloleysens/kibana that referenced this pull request Apr 22, 2020
…bana into pipeline-editor-part-mvp-2

* 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (34 commits)
  [Ingest Node Pipelines] Clone Pipeline (elastic#64049)
  Move authz lib out of snapshot restore (elastic#63947)
  Migrate vis_type_table to kibana/new platform  (elastic#63105)
  Enable include/exclude in Terms agg for numeric fields (elastic#59425)
  follow conventions for saved object definitions (elastic#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838)
  Fix page layouts, clean up unused code (elastic#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859)
  [Maps] Do not fetch geo_shape from docvalues (elastic#63997)
  Vega doc changes (elastic#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049)
  Add sub-menus to Resolver node (for 75% zoom) (elastic#63476)
  [FTR] Add test suite metrics tracking/output (elastic#62515)
  [Ingest pipelines] Delete pipeline (elastic#63635)
  [ML] Fixing single metric viewer page padding (elastic#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623)
  [Reporting] Config flag to escape formula CSV values (elastic#63645)
  [Metrics UI] Remove remaining field filtering (elastic#63398)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation release_note:skip Skip the PR/issue when compiling release notes v7.8.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants