Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add stricter geohash parsing #30376

Merged
merged 5 commits into from
May 7, 2018
Merged

Add stricter geohash parsing #30376

merged 5 commits into from
May 7, 2018

Conversation

imotov
Copy link
Contributor

@imotov imotov commented May 4, 2018

Adds verification that geohashes are not empty and contain only
valid characters. It fixes the issue when en empty geohash is
treated as [-180, -90] and geohashes with non-geohash character
are getting resolved into invalid coordinates.

Closes #23579

@imotov
Add stricter geohash parsing
0f48daa 
Adds verification that geohashes are not empty and contain only
valid characters.

Closes elastic#23579
@imotov imotov added >bug :Analytics/Geo Indexing, search aggregations of geo points and shapes v7.0.0 v6.4.0 labels May 4, 2018
@imotov imotov requested review from jpountz and nknize May 4, 2018 02:33
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

jpountz
jpountz approved these changes May 4, 2018
View reviewed changes
Copy link
Contributor

@jpountz jpountz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be considered a breaking change, but at the same time if someone relied on the leniency then it means that they were indexing corrupt data, so I don't think we should be concerned about it. So LGTM, and +1 to backport to 6.4.

server/src/main/java/org/elasticsearch/common/geo/GeoPoint.java
hash = mortonEncode(geohash);
} catch (IllegalArgumentException ex) {
throw new ElasticsearchParseException(ex.getMessage(), ex);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we just use the IAE?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All other formatting issues are throwing ElasticsearchParseException at the moment, so I have decided to stick with it for consistency sake. It also makes it easier to ignore in parseGeoPointIgnoringMalformed and parseGeoPointStringIgnoringMalformed later on if mapping has ignore_malformed set to true.

@imotov imotov merged commit 6fb189c into elastic:master May 7, 2018
jasontedor added a commit to jaymode/elasticsearch that referenced this pull request May 7, 2018
@jasontedor
Merge remote-tracking branch 'origin/master' into pr/30367
90a1f2c 
* origin/master: (39 commits)
  Docs: fix changelog merge
  Fix line length violation in cache tests
  Add stricter geohash parsing (elastic#30376)
  Add failing test for core cache deadlock
  [DOCS] convert forcemerge snippet
  Update forcemerge.asciidoc (elastic#30113)
  Added zentity to the list of API extension plugins (elastic#29143)
  Fix the search request default operation behavior doc (elastic#29302) (elastic#29405)
  Watcher: Mark watcher as started only after loading watches (elastic#30403)
  Pass the task to broadcast actions (elastic#29672)
  Disable REST default settings testing until elastic#29229 is back-ported
  Correct wording in log message (elastic#30336)
  Do not fail snapshot when deleting a missing snapshotted file (elastic#30332)
  AwaitsFix testCreateShrinkIndexToN
  DOCS: Correct mapping tags in put-template api
  DOCS: Fix broken link in the put index template api
  Add put index template api to high level rest client (elastic#30400)
  Relax testAckedIndexing to allow document updating
  [Docs] Add snippets for POS stop tags default value
  Move respect accept header on no handler to 6.3.1
  ...
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request May 7, 2018
@jasontedor
Merge branch 'master' into no-copy-settings-for-you
d756e66 
* master:
  Fix line length violation in cache tests
  Add stricter geohash parsing (elastic#30376)
  Add failing test for core cache deadlock
  [DOCS] convert forcemerge snippet
imotov added a commit that referenced this pull request May 7, 2018
@imotov
Add stricter geohash parsing (#30376)
58308a0 
Adds verification that geohashes are not empty and contain only
valid characters. It fixes the issue when en empty geohash is
treated as [-180, -90] and geohashes with non-geohash character
are getting resolved into invalid coordinates.

Closes #23579
@imotov imotov mentioned this pull request May 8, 2018
Closed
colings86 pushed a commit that referenced this pull request May 8, 2018
@imotov @colings86
Add stricter geohash parsing (#30376)
2518fef 
Adds verification that geohashes are not empty and contain only
valid characters. It fixes the issue when en empty geohash is
treated as [-180, -90] and geohashes with non-geohash character
are getting resolved into invalid coordinates.

Closes #23579
dnhatn added a commit that referenced this pull request May 8, 2018
@dnhatn
Merge branch 'master' into ccr
5d99157 
* master:
  Mute ML upgrade test (#30458)
  Stop forking javac (#30462)
  Client: Deprecate many argument performRequest (#30315)
  Docs: Use task_id in examples of tasks (#30436)
  Security: Rename IndexLifecycleManager to SecurityIndexManager (#30442)
  [Docs] Fix typo in cardinality-aggregation.asciidoc (#30434)
  Avoid NPE in `more_like_this` when field has zero tokens (#30365)
  Build: Switch to building javadoc with html5 (#30440)
  Add a quick tour of the project to CONTRIBUTING (#30187)
  Reindex: Use request flavored methods (#30317)
  Silence SplitIndexIT.testSplitIndexPrimaryTerm test failure. (#30432)
  Auto-expand replicas when adding or removing nodes (#30423)
  Docs: fix changelog merge
  Fix line length violation in cache tests
  Add stricter geohash parsing (#30376)
  Add failing test for core cache deadlock
  [DOCS] convert forcemerge snippet
  Update forcemerge.asciidoc (#30113)
  Added zentity to the list of API extension plugins (#29143)
  Fix the search request default operation behavior doc (#29302) (#29405)
dnhatn added a commit that referenced this pull request May 8, 2018
@dnhatn
Merge branch '6.x' into ccr-6.x
a5e4024 
* 6.x:
  Stop forking javac (#30462)
  Fix tribe tests
  Docs: Use task_id in examples of tasks (#30436)
  Security: Rename IndexLifecycleManager to SecurityIndexManager (#30442)
  Packaging: Set elasticsearch user to have non-existent homedir (#29007)
  [Docs] Fix typo in cardinality-aggregation.asciidoc (#30434)
  Avoid NPE in `more_like_this` when field has zero tokens (#30365)
  Build: Switch to building javadoc with html5 (#30440)
  Add a quick tour of the project to CONTRIBUTING (#30187)
  Add stricter geohash parsing (#30376)
  Reindex: Use request flavored methods (#30317)
  Silence SplitIndexIT.testSplitIndexPrimaryTerm test failure.  (#30432)
  Auto-expand replicas when adding or removing nodes (#30423)
  Silence IndexUpgradeIT test failures. (#30430)
  Fix line length violation in cache tests
  Add failing test for core cache deadlock
  [DOCS] convert forcemerge snippet
  Update forcemerge.asciidoc (#30113)
  Added zentity to the list of API extension plugins (#29143)
  Fix the search request default operation behavior doc (#29302) (#29405)
  Watcher: Mark watcher as started only after loading watches (#30403)
  Correct wording in log message (#30336)
  Do not fail snapshot when deleting a missing snapshotted file (#30332)
  AwaitsFix testCreateShrinkIndexToN
  DOCS: Correct mapping tags in put-template api
  DOCS: Fix broken link in the put index template api
  Add put index template api to high level rest client (#30400)
  [Docs] Add snippets for POS stop tags default value
  Remove entry inadvertently picked into changelog
  Move respect accept header on no handler to 6.3.1
  Respect accept header on no handler (#30383)
  [Test] Add analysis-nori plugin to the vagrant tests
  [Rollup] Validate timezone in range queries (#30338)
  [Docs] Fix bad link
  [Docs] Fix end of section in the korean plugin docs
  add the Korean nori plugin to the change logs
  Expose the Lucene Korean analyzer module in a plugin (#30397)
  Docs: remove transport_client from CCS role example (#30263)
  Test: remove cluster permission from CCS user (#30262)
  Watcher: Remove unneeded index deletion in tests
  fix docs branch version
  fix lucene snapshot version
  Upgrade to 7.4.0-snapshot-1ed95c097b (#30357)
  [ML][TEST] Clean up jobs in ModelPlotIT
  Watcher: Ensure trigger service pauses execution (#30363)
  [DOCS] Fixes ordering of changelog sections
  [DOCS] Commented out empty sections in the changelog to fix the doc build. (#30372)
  Make RepositoriesMetaData contents unmodifiable (#30361)
  Change signature of Get Repositories Response (#30333)
  6.x Backport: Terms query validate bug  (#30319)
  InternalEngineTests.testConcurrentOutOfOrderDocsOnReplica should use two documents (#30121)
  Security: reduce garbage during index resolution (#30180)
  Test: use trial license in qa tests with security
  [ML] Add integration test for model plots (#30359)
  SQL: Fix bug caused by empty composites (#30343)
  [ML] Account for gaps in data counts after job is reopened (#30294)
  [ML] Refactor DataStreamDiagnostics to use array (#30129)
  Make licensing FIPS-140 compliant (#30251)
  Do not load global state when deleting a snapshot (#29278)
  Don't load global state when only restoring indices (#29239)
  Tests: Use different watch ids per test in smoke test (#30331)
  Watcher: Make start/stop cycle more predictable and synchronous (#30118)
  [Docs] Add term query with normalizer example
  Adds Eclipse config for xpack licence headers (#30299)
  Fix message content in users tool (#30293)
  [DOCS] Removed X-Pack breaking changes page
  [DOCS] Added security breaking change
  [DOCS] Fixes link to TLS LDAP info
  [DOCS] Merges X-Pack release notes into changelog (#30350)
  [DOCS] Fixes broken links to bootstrap user (#30349)
  [Docs] Remove errant changelog line
  Fix NPE when CumulativeSum agg encounters null/empty bucket (#29641)
  [DOCS] Reorganizes authentication details in Stack Overview (#30280)
  Tests: Simplify VersionUtils released version splitting (#30322)
  Fix merging logic of Suggester Options (#29514)
  ReplicationTracker.markAllocationIdAsInSync may hang if allocation is cancelled (#30316)
  [DOCS] Adds LDAP realm configuration details (#30214)
  [DOCS] Adds native realm configuration details (#30215)
  Disable SSL on testing old BWC nodes (#30337)
  [DOCS] Enables edit links for X-Pack pages
  Cancelling a peer recovery on the source can leak a primary permit (#30318)
  SQL: Reduce number of ranges generated for comparisons (#30267)
  [DOCS] Adds links to changelog sections
  Convert server javadoc to html5 (#30279)
  REST Client: Add Request object flavored methods (#29623)
  Create default ES_TMPDIR on Windows (#30325)
  [Docs] Clarify `fuzzy_like_this` redirect (#30183)
  Fix docs of the `_ignored` meta field.
  Add a new `_ignored` meta field. (#29658)
  Move repository-azure fixture test to QA project (#30253)
@imotov imotov deleted the issue-23579-empty-geohashes branch May 1, 2020 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jpountz jpountz jpountz approved these changes

@nknize nknize Awaiting requested review from nknize

Assignees
No one assigned
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes >bug v6.4.0 v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Empty geo_point being treated as [-180, -90]?
4 participants
@imotov @elasticmachine @jpountz @colings86