Make licensing FIPS-140 compliant #30251

jkakavas · 2018-04-30T09:12:20Z

Necessary changes so that the licensing functionality can be
used in a FIPS-140 approved JVM.

Uses adequate salt length for content encryption
Changes key derivation to PBKDF2WithHmacSHA512 from a custom
approach with SHA512 and manual key stretching
Removes redundant manual padding

Other relevant changes:

Uses the SAH512 hash instead of the encrypted key bytes as the
key fingerprint to be included in the license specification
Removes the explicit verification check of the encryption key
as this is implicitly checked in signature verification.

Necessary changes so that the licensing functionality can be used in a FIPS 140 approved JVM. * Uses adequate salt length in encryption * Changes key derivation to PBKDF2WithHmacSHA512 from a custom approach with SHA512 and manual key stretching * Removes redundant manual padding Other relevant changes: * Uses the SAH512 hash instead of the encrypted key bytes as the key fingerprint to be included in the license specification * Removes the explicit verification check of the encryption key as this is implicitly checked in signature verification.

elasticmachine · 2018-04-30T20:52:19Z

Pinging @elastic/es-security

jaymode

I left a few minor comments, otherwise LGTM

jaymode · 2018-05-01T15:23:46Z

x-pack/license-tools/src/main/java/org/elasticsearch/license/licensor/LicenseSigner.java

@@ -59,9 +61,11 @@ public License sign(License licenseSpec) throws IOException {
                Collections.singletonMap(License.LICENSE_SPEC_VIEW_MODE, "true");
        licenseSpec.toXContent(contentBuilder, new ToXContent.MapParams(licenseSpecViewMode));
        final byte[] signedContent;
+        final boolean legacy = licenseSpec.version() < License.VERSION_CRYPTO_ALGORITHMS;


can we use preV4 instead of legacy here?

jaymode · 2018-05-01T15:24:48Z

x-pack/plugin/core/src/main/java/org/elasticsearch/license/CryptUtils.java

-    private static final byte[] salt = {
-            (byte) 0xA9, (byte) 0xA2, (byte) 0xB5, (byte) 0xDE,
-            (byte) 0x2A, (byte) 0x8A, (byte) 0x9A, (byte) 0xE6
+    // SALT must be at least 128bits


can you add for FIPS 140-2 compliance

jaymode · 2018-05-01T15:26:01Z

x-pack/plugin/core/src/main/java/org/elasticsearch/license/CryptUtils.java


+    static byte[] encryptV3Format(byte[] data) {
+        try {
+            SecretKey encryptionKey = getv3Key();


style wise, I think getV3Key reads easier

jaymode · 2018-05-01T15:27:07Z

x-pack/plugin/core/src/main/java/org/elasticsearch/license/LicenseVerifier.java

@@ -88,4 +83,5 @@ public static boolean verifyLicense(final License license) {
        }
        return verifyLicense(license, publicKeyBytes);
    }
+


nit: unneeded extra line

rjernst

Ditto Jay's comments about naming, otherwise LGTM

…-compliant

Necessary changes so that the licensing functionality can be used in a JVM in FIPS 140 approved mode. * Uses adequate salt length in encryption * Changes key derivation to PBKDF2WithHmacSHA512 from a custom approach with SHA512 and manual key stretching * Removes redundant manual padding Other relevant changes: * Uses the SAH512 hash instead of the encrypted key bytes as the key fingerprint to be included in the license specification * Removes the explicit verification check of the encryption key as this is implicitly checked in signature verification.

* master: Set the new lucene version for 6.4.0 [ML][TEST] Clean up jobs in ModelPlotIT Upgrade to 7.4.0-snapshot-1ed95c097b (#30357) Watcher: Ensure trigger service pauses execution (#30363) [DOCS] Added coming qualifiers in changelog [DOCS] Commented out empty sections in the changelog to fix the doc build. (#30372) Security: reduce garbage during index resolution (#30180) Make RepositoriesMetaData contents unmodifiable (#30361) Change quad tree max levels to 29. Closes #21191 (#29663) Test: use trial license in qa tests with security [ML] Add integration test for model plots (#30359) SQL: Fix bug caused by empty composites (#30343) [ML] Account for gaps in data counts after job is reopened (#30294) InternalEngineTests.testConcurrentOutOfOrderDocsOnReplica should use two documents (#30121) Change signature of Get Repositories Response (#30333) Tests: Use different watch ids per test in smoke test (#30331) [Docs] Add term query with normalizer example Adds Eclipse config for xpack licence headers (#30299) Watcher: Make start/stop cycle more predictable and synchronous (#30118) [test] add debug logging for packaging test [DOCS] Removed X-Pack Breaking Changes [DOCS] Fixes link to TLS LDAP info Update versions for start_trial after backport (#30218) Packaging: Set elasticsearch user to have non-existent homedir (#29007) [DOCS] Fixes broken links to bootstrap user (#30349) Fix NPE when CumulativeSum agg encounters null/empty bucket (#29641) Make licensing FIPS-140 compliant (#30251) [DOCS] Reorganizes authentication details in Stack Overview (#30280) Network: Remove http.enabled setting (#29601) Fix merging logic of Suggester Options (#29514) [DOCS] Adds LDAP realm configuration details (#30214) [DOCS] Adds native realm configuration details (#30215) ReplicationTracker.markAllocationIdAsInSync may hang if allocation is cancelled (#30316) [DOCS] Enables edit links for X-Pack pages (#30278) Packaging: Unmark systemd service file as a config file (#29004) SQL: Reduce number of ranges generated for comparisons (#30267) Tests: Simplify VersionUtils released version splitting (#30322) Cancelling a peer recovery on the source can leak a primary permit (#30318) Added changelog entry for deb prerelease version change (#30184) Convert server javadoc to html5 (#30279) Create default ES_TMPDIR on Windows (#30325) [Docs] Clarify `fuzzy_like_this` redirect (#30183) Post backport of #29658. Fix docs of the `_ignored` meta field. Remove MapperService#types(). (#29617) Remove useless version checks in REST tests. (#30165) Add a new `_ignored` meta field. (#29658) Move repository-azure fixture test to QA project (#30253) # Conflicts: # buildSrc/version.properties # server/src/test/java/org/elasticsearch/index/engine/InternalEngineTests.java

* 6.x: Stop forking javac (#30462) Fix tribe tests Docs: Use task_id in examples of tasks (#30436) Security: Rename IndexLifecycleManager to SecurityIndexManager (#30442) Packaging: Set elasticsearch user to have non-existent homedir (#29007) [Docs] Fix typo in cardinality-aggregation.asciidoc (#30434) Avoid NPE in `more_like_this` when field has zero tokens (#30365) Build: Switch to building javadoc with html5 (#30440) Add a quick tour of the project to CONTRIBUTING (#30187) Add stricter geohash parsing (#30376) Reindex: Use request flavored methods (#30317) Silence SplitIndexIT.testSplitIndexPrimaryTerm test failure. (#30432) Auto-expand replicas when adding or removing nodes (#30423) Silence IndexUpgradeIT test failures. (#30430) Fix line length violation in cache tests Add failing test for core cache deadlock [DOCS] convert forcemerge snippet Update forcemerge.asciidoc (#30113) Added zentity to the list of API extension plugins (#29143) Fix the search request default operation behavior doc (#29302) (#29405) Watcher: Mark watcher as started only after loading watches (#30403) Correct wording in log message (#30336) Do not fail snapshot when deleting a missing snapshotted file (#30332) AwaitsFix testCreateShrinkIndexToN DOCS: Correct mapping tags in put-template api DOCS: Fix broken link in the put index template api Add put index template api to high level rest client (#30400) [Docs] Add snippets for POS stop tags default value Remove entry inadvertently picked into changelog Move respect accept header on no handler to 6.3.1 Respect accept header on no handler (#30383) [Test] Add analysis-nori plugin to the vagrant tests [Rollup] Validate timezone in range queries (#30338) [Docs] Fix bad link [Docs] Fix end of section in the korean plugin docs add the Korean nori plugin to the change logs Expose the Lucene Korean analyzer module in a plugin (#30397) Docs: remove transport_client from CCS role example (#30263) Test: remove cluster permission from CCS user (#30262) Watcher: Remove unneeded index deletion in tests fix docs branch version fix lucene snapshot version Upgrade to 7.4.0-snapshot-1ed95c097b (#30357) [ML][TEST] Clean up jobs in ModelPlotIT Watcher: Ensure trigger service pauses execution (#30363) [DOCS] Fixes ordering of changelog sections [DOCS] Commented out empty sections in the changelog to fix the doc build. (#30372) Make RepositoriesMetaData contents unmodifiable (#30361) Change signature of Get Repositories Response (#30333) 6.x Backport: Terms query validate bug (#30319) InternalEngineTests.testConcurrentOutOfOrderDocsOnReplica should use two documents (#30121) Security: reduce garbage during index resolution (#30180) Test: use trial license in qa tests with security [ML] Add integration test for model plots (#30359) SQL: Fix bug caused by empty composites (#30343) [ML] Account for gaps in data counts after job is reopened (#30294) [ML] Refactor DataStreamDiagnostics to use array (#30129) Make licensing FIPS-140 compliant (#30251) Do not load global state when deleting a snapshot (#29278) Don't load global state when only restoring indices (#29239) Tests: Use different watch ids per test in smoke test (#30331) Watcher: Make start/stop cycle more predictable and synchronous (#30118) [Docs] Add term query with normalizer example Adds Eclipse config for xpack licence headers (#30299) Fix message content in users tool (#30293) [DOCS] Removed X-Pack breaking changes page [DOCS] Added security breaking change [DOCS] Fixes link to TLS LDAP info [DOCS] Merges X-Pack release notes into changelog (#30350) [DOCS] Fixes broken links to bootstrap user (#30349) [Docs] Remove errant changelog line Fix NPE when CumulativeSum agg encounters null/empty bucket (#29641) [DOCS] Reorganizes authentication details in Stack Overview (#30280) Tests: Simplify VersionUtils released version splitting (#30322) Fix merging logic of Suggester Options (#29514) ReplicationTracker.markAllocationIdAsInSync may hang if allocation is cancelled (#30316) [DOCS] Adds LDAP realm configuration details (#30214) [DOCS] Adds native realm configuration details (#30215) Disable SSL on testing old BWC nodes (#30337) [DOCS] Enables edit links for X-Pack pages Cancelling a peer recovery on the source can leak a primary permit (#30318) SQL: Reduce number of ranges generated for comparisons (#30267) [DOCS] Adds links to changelog sections Convert server javadoc to html5 (#30279) REST Client: Add Request object flavored methods (#29623) Create default ES_TMPDIR on Windows (#30325) [Docs] Clarify `fuzzy_like_this` redirect (#30183) Fix docs of the `_ignored` meta field. Add a new `_ignored` meta field. (#29658) Move repository-azure fixture test to QA project (#30253)

The errors were caused because release tests would use a copy of the public key that was formatted differently. The change to the public key format was introduced in [1]. Release tests Jenkins job has now been updated to use the correct key format depending on the branch they run on [2] Closes #30430 [1] #30251 [2] elastic/infra#4944

Allows rolling restart from 6.3 to 6.4. Relates to #30731 and #30251

jkakavas added review v7.0.0 :Security/Security Security issues without another label v6.4.0 labels Apr 30, 2018

jkakavas requested review from rjernst and jaymode April 30, 2018 09:12

hub-cap added :Security/Security Security issues without another label and removed :Security/Security Security issues without another label labels Apr 30, 2018

jaymode approved these changes May 1, 2018

View reviewed changes

jaymode added the >enhancement label May 1, 2018

rjernst approved these changes May 1, 2018

View reviewed changes

jkakavas added 2 commits May 2, 2018 13:57

Merge remote-tracking branch 'origin/master' into make-licensing-fips…

12c7d71

…-compliant

Address feedback

8ce03f4

jkakavas merged commit cca1a2a into elastic:master May 2, 2018

jkakavas added backport pending and removed review labels May 2, 2018

elasticmachine mentioned this pull request May 3, 2018

Security: FIPS 140-2 Compliance roadmap #29851

Closed

12 tasks

jkakavas removed the backport pending label May 3, 2018

This was referenced May 25, 2018

Enable rolling upgrades from default distribution prior to 6.3.0 to default distribution post 6.3.0 #30731

Closed

Only auto-update license signature if all nodes ready #30859

Merged

ywelsch added a commit that referenced this pull request Jun 5, 2018

Only auto-update license signature if all nodes ready (#30859)

3b98c26

Allows rolling restart from 6.3 to 6.4. Relates to #30731 and #30251

ywelsch added a commit that referenced this pull request Jun 5, 2018

Only auto-update license signature if all nodes ready (#30859)

97b3b2d

Allows rolling restart from 6.3 to 6.4. Relates to #30731 and #30251

jkakavas deleted the make-licensing-fips-compliant branch September 14, 2018 06:58

jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make licensing FIPS-140 compliant #30251

Make licensing FIPS-140 compliant #30251

jkakavas commented Apr 30, 2018

elasticmachine commented Apr 30, 2018

jaymode left a comment

jaymode May 1, 2018

jaymode May 1, 2018

jaymode May 1, 2018

jaymode May 1, 2018

rjernst left a comment

Make licensing FIPS-140 compliant #30251

Make licensing FIPS-140 compliant #30251

Conversation

jkakavas commented Apr 30, 2018

elasticmachine commented Apr 30, 2018

jaymode left a comment

Choose a reason for hiding this comment

jaymode May 1, 2018

Choose a reason for hiding this comment

jaymode May 1, 2018

Choose a reason for hiding this comment

jaymode May 1, 2018

Choose a reason for hiding this comment

jaymode May 1, 2018

Choose a reason for hiding this comment

rjernst left a comment

Choose a reason for hiding this comment