Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added OpenSearch 3.0. #459

Merged
merged 7 commits into from
Aug 4, 2024
Merged

Conversation

dblock
Copy link
Member

@dblock dblock commented Aug 1, 2024

Description

  • Adds OpenSearch 3.0 to CI/CD.
  • Dumps docker logs in case tests fail to debug server-side failures.
  • Increases memory limit threshold for SQL that fails intermittently.
  • Removes unneeded quotes in docker compose.

Issues Resolved

N/A

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

github-actions bot commented Aug 1, 2024

Changes Analysis

Commit SHA: d909cb5
Comparing To SHA: 8bf497c

API Changes

Summary

└─┬Components
  └─┬nodes._common:ShardSearchBackpressureTaskStats
    └──[➕] properties (46405:9)

Document Element Total Changes Breaking Changes
components 1 0
  • Total Changes: 1
  • Additions: 1

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/10228661142/artifacts/1772277099

API Coverage

Before After Δ
Covered (%) 510 (49.95 %) 510 (49.95 %) 0 (0 %)
Uncovered (%) 511 (50.05 %) 511 (50.05 %) 0 (0 %)
Unknown 24 24 0

Copy link
Contributor

github-actions bot commented Aug 1, 2024

Spec Test Coverage Analysis

Total Tested
534 118 (22.1 %)

@dblock
Copy link
Member Author

dblock commented Aug 1, 2024

The intermittent SQL failure is an OOM.

2024-08-01T14:02:08.9247018Z [2024-08-01T14:02:05,858][INFO ][o.o.s.s.a.SQLSyntaxParser] [6080215090a3] New Engine Request Query: ( SELECT * FROM table )
2024-08-01T14:02:08.9247664Z [2024-08-01T14:02:05,983][WARN ][o.o.s.o.m.OpenSearchMemoryHealthy] [6080215090a3] Memory usage:1054384512 exceed limit:912680550
2024-08-01T14:02:08.9248242Z [2024-08-01T14:02:05,983][WARN ][o.o.s.o.m.OpenSearchMemoryHealthy] [6080215090a3] Fast failure the current request
2024-08-01T14:02:08.9249007Z [2024-08-01T14:02:05,985][ERROR][o.o.s.l.p.RestSqlAction  ] [6080215090a3] ef30b895-53e0-4444-8034-3480c19b181c Server side error during query execution
2024-08-01T14:02:08.9249390Z java.lang.IllegalStateException: resource is not enough to run the query, quit.
2024-08-01T14:02:08.9250329Z 	at org.opensearch.sql.opensearch.executor.protector.ResourceMonitorPlan.open(ResourceMonitorPlan.java:47) ~[opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9250707Z 	at java.base/java.util.Collections$SingletonList.forEach(Collections.java:5186) ~[?:?]
2024-08-01T14:02:08.9251234Z 	at org.opensearch.sql.planner.physical.PhysicalPlan.open(PhysicalPlan.java:29) ~[core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9252023Z 	at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$0(OpenSearchExecutionEngine.java:49) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9252708Z 	at org.opensearch.sql.opensearch.client.OpenSearchNodeClient.schedule(OpenSearchNodeClient.java:177) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9253640Z 	at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.execute(OpenSearchExecutionEngine.java:43) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9254412Z 	at org.opensearch.sql.executor.QueryService.lambda$executePlan$1(QueryService.java:66) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9254706Z 	at java.base/java.util.Optional.ifPresentOrElse(Optional.java:198) [?:?]
2024-08-01T14:02:08.9255209Z 	at org.opensearch.sql.executor.QueryService.executePlan(QueryService.java:63) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9255688Z 	at org.opensearch.sql.executor.QueryService.execute(QueryService.java:41) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9256179Z 	at org.opensearch.sql.executor.execution.QueryPlan.execute(QueryPlan.java:63) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9256928Z 	at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$submit$0(OpenSearchQueryManager.java:31) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9257734Z 	at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$withCurrentContext$1(OpenSearchQueryManager.java:45) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9258452Z 	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:882) [opensearch-2.15.0.jar:2.15.0]
2024-08-01T14:02:08.9258901Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
2024-08-01T14:02:08.9259319Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
2024-08-01T14:02:08.9259532Z 	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]

@dblock dblock marked this pull request as ready for review August 1, 2024 15:13
@dblock dblock force-pushed the opensearch-3.0 branch 6 times, most recently from c272e84 to 36ae2ac Compare August 2, 2024 19:09
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
@Xtansia Xtansia merged commit 13a603c into opensearch-project:main Aug 4, 2024
13 checks passed
@dblock dblock deleted the opensearch-3.0 branch August 4, 2024 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants