Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] BuildExamplePluginsIT testCurrentExamplePlugin fails in FIPS JVM #39855

Closed
droberts195 opened this issue Mar 8, 2019 · 5 comments · Fixed by #39917
Closed

[CI] BuildExamplePluginsIT testCurrentExamplePlugin fails in FIPS JVM #39855

droberts195 opened this issue Mar 8, 2019 · 5 comments · Fixed by #39917
Assignees
@tvernum
Labels
:Security/Security Security issues without another label >test-failure Triaged test failures from CI

Comments

@droberts195
Copy link
Contributor

BuildExamplePluginsIT testCurrentExamplePlugin was failing globally for a long time, but most problems were fixed in #38899. However, it still consistently fails in the FIPS JVM:

The error is:

    Execution failed for task ':integTestCluster#wait'.
    > Failed to locate seed node transport file [/tmp/junit7423325524789213283/build/cluster/integTestCluster node0/elasticsearch-8.0.0-SNAPSHOT/logs/transport.ports]: timed out waiting for it to be created after 40 seconds
@droberts195 droberts195 added >test-failure Triaged test failures from CI :Security/Security Security issues without another label labels Mar 8, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security

@tvernum tvernum self-assigned this Mar 11, 2019
@tvernum
Copy link
Contributor

tvernum commented Mar 11, 2019

That the build is failing on this integration test is coincidental. It's actually failing on all integration tests due to #38013.

I need to look into exactly why.

@tvernum
Copy link
Contributor

tvernum commented Mar 11, 2019 •
edited
Loading

I have tracked it down, and it's actually a pretty big deal.

It turns out that up until #38013 and #39732, the ClusterFormationTasks would use JAVA_HOME (not compiler.java or runtime.java) to run the integration test cluster.

That means our integTest tasks have never run on a FIPS JVM until now. And they don't work.

We now respect runtime.java for running these test clusters (which is a Good Thing :TM:), but we've got some work to do to make it run successfully.

@tvernum
Copy link
Contributor

tvernum commented Mar 11, 2019

If we get agreement that we want to keep using runtime.java, I think it might be worth closing this issue, and opening a general "Make Integration tests work on FIPS" issue.

@danielmitterdorfer
Copy link
Member

There have been ~ 30 more of these test failures over the weekend; mentioning each and every one of them here will just add noise IMHO but the failure can be reproduced with the following reproduction line (assuming a FIPS JVM is installed at $JAVA8FIPS_HOME):

export RUNTIME_JAVA_HOME=$JAVA8FIPS_HOME
./gradlew :example-plugins:painless-whitelist:integTest

I agree with @tvernum's proposal that we should aim to make our integrations work in a FIPS JVM. However, I do wonder whether for the time being we want to implement a workaround to restore the old behavior to avoid spurious build failures.

@jkakavas jkakavas mentioned this issue Mar 11, 2019
Merged
danielmitterdorfer pushed a commit that referenced this issue Mar 11, 2019
@jkakavas @danielmitterdorfer
Allow IntegTestClusters to run with FIPS 140 JVMs (#39917)
0e1cf3e 
The changes in #39732 mean that nodes in the IntegTest clusters will
now run with whichever java version is defined as `runtime.java` and
not JAVA_HOME anymore.
This means that these nodes will also run in JVM with fips approved
mode enabled and as such, need to have access to the password for the
BCFKS keystore that is used as the default keystore/truststore.

This change sets the two necessary system properties.

Resolves #39855
danielmitterdorfer pushed a commit to danielmitterdorfer/elasticsearch that referenced this issue Mar 11, 2019
@jkakavas @danielmitterdorfer
Allow IntegTestClusters to run with FIPS 140 JVMs (elastic#39917)
91a8dce 
The changes in elastic#39732 mean that nodes in the IntegTest clusters will
now run with whichever java version is defined as `runtime.java` and
not JAVA_HOME anymore.
This means that these nodes will also run in JVM with fips approved
mode enabled and as such, need to have access to the password for the
BCFKS keystore that is used as the default keystore/truststore.

This change sets the two necessary system properties.

Resolves elastic#39855
@danielmitterdorfer danielmitterdorfer mentioned this issue Mar 11, 2019
Merged
danielmitterdorfer pushed a commit to danielmitterdorfer/elasticsearch that referenced this issue Mar 11, 2019
@jkakavas @danielmitterdorfer
Allow IntegTestClusters to run with FIPS 140 JVMs (elastic#39917)
03efd03 
The changes in elastic#39732 mean that nodes in the IntegTest clusters will
now run with whichever java version is defined as `runtime.java` and
not JAVA_HOME anymore.
This means that these nodes will also run in JVM with fips approved
mode enabled and as such, need to have access to the password for the
BCFKS keystore that is used as the default keystore/truststore.

This change sets the two necessary system properties.

Resolves elastic#39855
@danielmitterdorfer danielmitterdorfer mentioned this issue Mar 11, 2019
Merged
danielmitterdorfer added a commit that referenced this issue Mar 12, 2019
@danielmitterdorfer
Allow IntegTestClusters to run with FIPS 140 JVMs (#39917) (#39924)
da1e5cd 
The changes in #39732 mean that nodes in the IntegTest clusters will
now run with whichever java version is defined as `runtime.java` and
not JAVA_HOME anymore.
This means that these nodes will also run in JVM with fips approved
mode enabled and as such, need to have access to the password for the
BCFKS keystore that is used as the default keystore/truststore.

This change sets the two necessary system properties.

Resolves #39855
danielmitterdorfer added a commit that referenced this issue Mar 12, 2019
@danielmitterdorfer
Allow IntegTestClusters to run with FIPS 140 JVMs (#39917) (#39923)
3400610 
The changes in #39732 mean that nodes in the IntegTest clusters will
now run with whichever java version is defined as `runtime.java` and
not JAVA_HOME anymore.
This means that these nodes will also run in JVM with fips approved
mode enabled and as such, need to have access to the password for the
BCFKS keystore that is used as the default keystore/truststore.

This change sets the two necessary system properties.

Resolves #39855
@jkakavas jkakavas mentioned this issue Mar 13, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tvernum tvernum

Labels
:Security/Security Security issues without another label >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow IntegTestClusters to run with FIPS 140 JVM jkakavas/elasticsearch
4 participants
@danielmitterdorfer @tvernum @droberts195 @elasticmachine