Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] FullClusterRestartIT testApiKeySuperuser failing for pre 7.3.0 old clusters #82785

Closed
jkakavas opened this issue Jan 19, 2022 · 4 comments · Fixed by #82792
Closed

[CI] FullClusterRestartIT testApiKeySuperuser failing for pre 7.3.0 old clusters #82785

jkakavas opened this issue Jan 19, 2022 · 4 comments · Fixed by #82792
Assignees
@ywangd
Labels
:Security/Authentication Logging in, Usernames/passwords, Realms (Native/LDAP/AD/SAML/PKI/etc) Team:Security Meta label for security team >test-failure Triaged test failures from CI

Comments

@jkakavas
Copy link
Member

Before 7.3.0, the role_descriptors field was mandatory in the create api key request ( #43481 ).

This test creates an API key with no explicitly set role descriptors so this request fails when run against a pre 7.3.0 old node. We can adjust the request depending on the version or run the tests only for >= 7.3.0 without losing much coverage in my opinion

Build scan:
https://gradle-enterprise.elastic.co/s/gkezc34izxkn2/tests/:x-pack:qa:full-cluster-restart:v7.0.0%23oldClusterTest/org.elasticsearch.xpack.restart.FullClusterRestartIT/testApiKeySuperuser

Reproduction line:
./gradlew ':x-pack:qa:full-cluster-restart:v7.0.0#oldClusterTest' -Dtests.class="org.elasticsearch.xpack.restart.FullClusterRestartIT" -Dtests.method="testApiKeySuperuser" -Dtests.seed=D55A379AD8E9B4D3 -Dtests.bwc=true -Dtests.locale=en-AU -Dtests.timezone=America/Araguaina -Druntime.java=17

Applicable branches:
8.0

Reproduces locally?:
Yes

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.xpack.restart.FullClusterRestartIT&tests.test=testApiKeySuperuser

Failure excerpt:

org.elasticsearch.client.ResponseException: method [PUT], host [http://127.0.0.1:35813], URI [/_security/api_key], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Required [role_descriptors]"}],"type":"illegal_argument_exception","reason":"Required [role_descriptors]"},"status":400}

  at __randomizedtesting.SeedInfo.seed([D55A379AD8E9B4D3:E2975A0EEB293500]:0)
  at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:346)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:312)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:287)
  at org.elasticsearch.xpack.restart.FullClusterRestartIT.testApiKeySuperuser(FullClusterRestartIT.java:360)
  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2)
  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:568)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:375)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:824)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:475)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:375)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:831)
  at java.lang.Thread.run(Thread.java:833)
@jkakavas jkakavas added :Security/Authentication Logging in, Usernames/passwords, Realms (Native/LDAP/AD/SAML/PKI/etc) >test-failure Triaged test failures from CI labels Jan 19, 2022
@elasticmachine elasticmachine added the Team:Security Meta label for security team label Jan 19, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security (Team:Security)

ywangd added a commit to ywangd/elasticsearch that referenced this issue Jan 19, 2022
@ywangd
[Test] provide role descriptors for API key creation
ebaa1ca 
The role descriptors became optional since version 7.3.0. For earlier
versions, they must be specified. This PR specifies them conditionally
based on the old cluster version. This also serves a variation of the
test to show that dropping write access to system indices from the
limiting role will prevent the key from writing to system indices as a
whole.

Resolves: elastic#82785
@ywangd ywangd mentioned this issue Jan 19, 2022
Merged
@ywangd
Copy link
Member

ywangd commented Jan 19, 2022

Thanks @jkakavas It's the second time I forgot this difference of old versions ... I raised #82792

@mark-vieira
Copy link
Contributor

mark-vieira commented Jan 19, 2022 •
edited
Loading

FWIW, I've seen this fail on cluster versions as new as 7.9 although not sure if it's the same root cause.

https://gradle-enterprise.elastic.co/s/tzo2kkyj2f2m6/tests/:x-pack:qa:full-cluster-restart:v7.9.0%23upgradedClusterTest/org.elasticsearch.xpack.restart.FullClusterRestartIT/testApiKeySuperuser?top-execution=1

@ywangd
Copy link
Member

ywangd commented Jan 19, 2022

Thanks Mark. It is not the same root cause but a genuine failure nonetheless. System indices deprecation warning was added in 7.10.0. So the warning expectation should be relaxed. I'll adjust the associated PR to fix this failure as well.

ywangd added a commit that referenced this issue Jan 20, 2022
@ywangd
[Test Mute] AwaitsFix #82785
6b6f06e
ywangd added a commit that referenced this issue Jan 20, 2022
@ywangd
[Test Mute] AwaitsFix #82785
b33768d
ywangd added a commit that referenced this issue Jan 24, 2022
@ywangd
[Test] Fix superuser API key bwc test (#82792)
43ea63b 
The role descriptors became optional since version 7.3.0. For earlier
versions, they must be specified. This PR specifies them conditionally
based on the old cluster version. This also serves a variation of the
test to show that dropping write access to system indices from the
limiting role will prevent the key from writing to system indices as a
whole.

Resolves: #82785
ywangd added a commit to ywangd/elasticsearch that referenced this issue Jan 24, 2022
@ywangd
[Test] Fix superuser API key bwc test (elastic#82792)
a0766f7 
The role descriptors became optional since version 7.3.0. For earlier
versions, they must be specified. This PR specifies them conditionally
based on the old cluster version. This also serves a variation of the
test to show that dropping write access to system indices from the
limiting role will prevent the key from writing to system indices as a
whole.

Resolves: elastic#82785
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ywangd ywangd

Labels
:Security/Authentication Logging in, Usernames/passwords, Realms (Native/LDAP/AD/SAML/PKI/etc) Team:Security Meta label for security team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Test] Fix superuser API key bwc test ywangd/elasticsearch
4 participants
@ywangd @mark-vieira @jkakavas @elasticmachine