Unable to load crypto library with otp version 26.2.5 when built erlang with fips-enabled

[HarinadhD]

I am trying to upgrade erlang version from 25.1.2 to 26.2.5 , erlang 26.X require to build rabbirmq-server 3.13.X

But got an error "Unable to load crypto library" when executing crypto:version(). in erl shell with OTP version 26.2.5

This problem is noticed with 26.2.5 , same issue was not observed with erlang 25.1.2
openssl installed version in both the cases 3.0.13

OTP 26.2.5:

root@vm [ws ]# erl
Erlang/OTP 26 [erts-14.2.5] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]
Eshell V14.2.5 (press Ctrl+G to abort, type help(). for help)
1> crypto:version().
=ERROR REPORT==== 11-Jun-2024::12:45:57.772369 ===
Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."
=WARNING REPORT==== 11-Jun-2024::12:45:57.777686 ===
The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}
** exception error: undefined function crypto:version/0
2

### NOTE:
The above issue got resolved after installing openssl-fips-provider package , but OTP 25.1.2 was working without openssl-fips-provider package.

OTP 25.1.2:

root@photon4 [ ~ ]# erl
Erlang/OTP 25 [erts-13.1.2] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]
Eshell V13.1.2 (abort with ^G)
1> crypto:version().
"5.1.2"
2>

To Reproduce
install openssl-devel package
build OTP with fips enabled
execute crypto:version().

root@vm[/ws/ ]# erl
Erlang/OTP 26 [erts-14.2.5] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]

Eshell V14.2.5 (press Ctrl+G to abort, type help(). for help)
1> crypto:version().
=ERROR REPORT==== 11-Jun-2024::12:45:57.772369 ===
Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."

=WARNING REPORT==== 11-Jun-2024::12:45:57.777686 ===
The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}

** exception error: undefined function crypto:version/0

Expected behavior
erlang should load crypto library without installing openssl-fips-provider package.

Affected versions
verified Only with OTP 26.2.5

[sshedi]

Following patch fixes the issue. (Patch is generated from 26.2.5 version of erlang).

---
diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c
@@ -224,7 +224,9 @@ static int initialize(ErlNifEnv* env, ERL_NIF_TERM load_info)
 #ifdef HAS_3_0_API
     prov_cnt = 0;
 # ifdef FIPS_SUPPORT
-    if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "fips"))) return __LINE__;
+    if (FIPS_MODE()) {
+        if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "fips"))) return __LINE__;
+    }
 #endif
     if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "default"))) return __LINE__;
     if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "base"))) return __LINE__;
--

This patch makes erlang load "fips.so" library only when openssl fips is enabled in the system.
Distros like Fedora have fips.so available with openssl-libs package.
In some of the distros where fips.so is not available by default (PhotonOS, Ubuntu for example), crypto module doesn't load at all because fips.so is not present, hence the error. Loading fips.so only when fips enabled seems to be the right way to fix this issue.

Can someone from erlang upstream please take this patch? I can't send a PR due to some restrictions at my end. Sorry about that.

[nickva]

I think I see the same issue when building with --enable-fips with Erlang 26 on Debian Bookworm.

~/otp_src_26.2.5.2$ openssl version
OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)

./bin/cerl 
Erlang/OTP 26 [erts-14.2.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Eshell V14.2.5.2 (press Ctrl+G to abort, type help(). for help)
1> crypto:strong_rand_bytes(1).
=ERROR REPORT==== 11-Jul-2024::19:02:53.521863 ===
Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."

=WARNING REPORT==== 11-Jul-2024::19:02:53.525380 ===
The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}

** exception error: undefined function crypto:strong_rand_bytes/1
2> 

The build string is:

./configure 
   --without-javac --without-wx --without-odbc  --without-debugger 
   --without-observer --without-et  --without-cosEvent --without-cosEventDomain 
   --without-cosFileTransfer  --without-cosNotification --without-cosProperty 
   --without-cosTime --without-cosTransactions --without-orber
   --enable-fips

If I drop --enable-fips it seems to work.

Was wondering what is the intent behind enable-fips option? Maybe we're using it incorrectly? To me seems it should be adding the ability to enable FIPS after the VM is compiled using -crypto fips_mode true|false but by default for false the system should still be able to call crypto:strong_rand_bytes/1?

[sverker]

Pull request #8762 is a proposed fix to this problem. I don't think the patch by @sshedi above is correct in the case when fips is provided.

#8762 is based on maint branch (OTP 27).
To test the same fix for OTP 26.2.5 use branch https://github.com/sverker/otp/tree/sverker/26/crypto/fips-config-without-fips-provider/OTP-19212.

[matijavibe]

@HarinadhD hey, I'm facing the same issue on ubuntu 22.04, erlang 26.1.2, you said that it was resolved after installing the openssl-fips-provider package, did you follow these instructions?
openssl-fips-provider

@sverker do you have an idea when approximately your fix will be released as a patch version for OTP 26?

[sverker]

@matijavibe It just missed OTP-26.2.5.3 and is now in the pipe for the next 26 patch release which has no set date but usually happen when a paying customer needs a fix, or we think there are enough important fixes to justify a tagged release.

[matijavibe]

Understood, 🤞 that a paying customer will require something soon, thanks for the response!! :)

[sverker]

Fixed in OTP-26.2.5.4.

[nickva]

Thank you very much for fixing the fips bug, Sverker!