Load Keystore Intelligently in the Driver #14

Syed-SnapLogic · 2021-03-15T09:53:49Z

Driver version

2.0.0.0

Redshift version

PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.24421

Client Operating System

Ubuntu 18.04

JAVA/JVM version

openjdk version "11.0.8" 2020-07-14

Table schema

Not Applicable

Problem description

- In your code, you are loading "cacerts" but the keystore is initialised using default type as per https://github.com/aws/amazon-redshift-jdbc-driver/blob/master/src/main/java/com/amazon/redshift/ssl/LibPQFactory.java#L370.
- In JDK8, default keystore type is "JKS" while in JDK11, its "PKCS12".
- And I am using BouncyCastle as the first provider in my java client program.
- And BouncyCastle is very strict in type, and I am getting the error "Error loading the provided keystore" because "cacerts" is of type "JKS" not "PKCS12".
- And if I remove BouncyCastle provider, then the default SUN provider primarily attempts to load "cacerts" as PKCS12 but if that fails, then the provider reattempts its as "JKS".  This is evident from the code https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/master/src/java.base/share/classes/sun/security/pkcs12/PKCS12KeyStore.java#L140.
- Bottom line: your code doesn't work with BouncyCastle provider.

JDBC trace logs

Not Needed

Reproduction code

- Take the source code at https://github.com/Syed-SnapLogic/SyedsPublicRepo/tree/master/bctestcacerts
- Provide your cacerts file from any JDK you have.
- You can change the passphrase to null and try.

Recommendation

Why don't you load the keystore as follows:
try {
KeyStore keyStore = KeyStore.getInstance("JKS");
keyStore.load(inputStream, passphrase);
// do rest of the logic
} catch (Exception e1) {
try {
KeyStore keyStore = KeyStore.getInstance("PKCS12");
keyStore.load(inputStream, passphrase);
// do rest of the logic
} catch (Exception e2) {
try {
KeyStore keyStore = KeyStore.getInstance("JCEKS");
keyStore.load(inputStream, passphrase);
// do rest of the logic
} catch (Exception e1) {
}
}
}
Note: You have the recreate the inputStream before retrying.

The text was updated successfully, but these errors were encountered:

iggarish · 2021-03-15T20:49:25Z

Thanks for reporting issue. We will look into it.

iggarish · 2021-03-16T00:26:06Z

Hi Syed,

Could you please try using "sslkey" connection option? If this file ends with pk8 then we use JKS, if it ends with p12 then we use PKCS12 key store. Also generate JDBC driver logs using "LogLevel=6;LogPath=my_path" and run your test and send us *.log from my_path directory.

Thanks,
ilesh Garish

Syed-SnapLogic · 2021-03-16T03:59:32Z

@iggarish , you guys are loading "cacerts" that gets bundled with JDK. I don't have any ssl key with me.
Its while loading the "cacerts" that we are getting this error because you are loading cacerts as PKCS12 instead of as JKS. The fix has to be in your code. You can try the following safe or fallback approach:

try {
  load keystore as pkcs12;
} catch (Exception e1) {
  try {
    load keystore as jks;
  } catch (Exception e2) {
    try {
      load keystore as jceks;
    } catch (Exception e3) {
      throw keystore error;
    }
 }
}

iggarish · 2021-03-16T22:39:37Z

Hi Syed,
I clone your repo and try to reproduce the issue. I did following steps:

Change Loader.java:
public static final String cacerts = "/home/iggarish/jdk1.8.0_241/jre/lib/security/cacerts";
public static final String passphrase = null;
mvn clean install
cd target
dev-dsk-iggarish-2a-77ebad06 % java -cp /home/iggarish/.m2/repository/org/bouncycastle/bcpg-jdk15on/1.68/bcpg-jdk15on-1.68.jar:/home/iggarish/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.68/bcprov-jdk15on-1.68.jar:/home/iggarish/.m2/repository/org/bouncycastle/bcprov-ext-jdk15on/1.68/bcprov-ext-jdk15on-1.68.jar:/home/iggarish/.m2/repository/org/bouncycastle/bctls-jdk15on/1.68/bctls-jdk15on-1.68.jar:./bctestcacerts-1.0-SNAPSHOT.jar com.syed.bctestcacerts.Loader
BC
BCJSSE
SUN
SunRsaSign
SunEC
SunJSSE
SunJCE
SunJGSS
SunSASL
XMLDSig
SunPCSC
type: jks
java.security.KeyStore
successful!!

One think I notice you have JDK11 and I am using JDK8. Am I missing anything? Do you have steps with any sample JDBC program, so I can see the issue? I get you what you are trying to say. But I need to reproduce the issue, fix it and then test it again.

iggarish · 2021-03-16T23:09:33Z

Hi Syed,

I add some code (as per your LoadJava) in my JDBC test to add Bouncy Castle provider and then connect. It works fine.
BC
BCJSSE
SUN
SunRsaSign
SunEC
SunJSSE
SunJCE
SunJGSS
SunSASL
XMLDSig
SunPCSC
type: jks
Mar 16, 2021 11:05:27 PM org.bouncycastle.jsse.provider.PropertyUtils getStringSecurityProperty
INFO: Found string security property [jdk.tls.disabledAlgorithms]: SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL
Mar 16, 2021 11:05:27 PM org.bouncycastle.jsse.provider.PropertyUtils getStringSecurityProperty
INFO: Found string security property [jdk.certpath.disabledAlgorithms]: MD2, MD5, SHA1 jdkCA & usage TLSServer, RSA keySize < 1024, DSA keySize < 1024, EC keySize < 224
Mar 16, 2021 11:05:27 PM org.bouncycastle.jsse.provider.DisabledAlgorithmConstraints create
WARNING: Ignoring unsupported entry in 'jdk.certpath.disabledAlgorithms': SHA1 jdkCA & usage TLSServer
Mar 16, 2021 11:05:27 PM org.bouncycastle.jsse.provider.PropertyUtils getBooleanSecurityProperty
INFO: Found boolean security property [keystore.type.compat]: true
Mar 16, 2021 11:05:27 PM org.bouncycastle.jsse.provider.ProvTrustManagerFactorySpi getDefaultTrustStore
INFO: Initializing with trust store at path: /local/home/iggarish/jdk1.8.0_241/jre/lib/security/cacerts
Connected... Driver Version: 2.0.0.0

Note that I am using JDK 8.0.

This is function I created in my JDBC test connect program:
private void addBouncyCastleProvider() {
Security.insertProviderAt(new BouncyCastleProvider(), 1);
Security.insertProviderAt(new BouncyCastleJsseProvider(), 2);
Provider[] p = Security.getProviders();
for (Provider P : p) {
System.out.println(P.getName());
}
System.out.println("type: " + KeyStore.getDefaultType());
}
...

		ts.addBouncyCastleProvider();
		connection = ts.getIAMConnection(props, IAM_URL, "test");

....

iggarish · 2021-03-16T23:25:27Z

I am downloading jdk11.0.8 and trying it.

iggarish · 2021-03-16T23:37:02Z

I tried my test program but looks like something more missing with JDK11:
dev-dsk-iggarish-2a-77ebad06 % java TestIAMUserConnect
java: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /local/home/iggarish/jdk11.0.8/openlogic-openjdk-11.0.8+10-linux-x64/bin/../lib/jli/libjli.so)

iggarish · 2021-03-16T23:56:40Z

Ok. I downloaded another JDK11 and I can see the issue as follow:
dev-dsk-iggarish-2a-77ebad06 % java TestIAMUserConnectWithBouncyCastle
BC
BCJSSE
SUN
SunRsaSign
SunEC
SunJSSE
SunJCE
SunJGSS
SunSASL
XMLDSig
SunPCSC
JdkLDAP
JdkSASL
SunPKCS11
type: pkcs12
Mar 16, 2021 11:55:45 PM org.bouncycastle.jsse.provider.PropertyUtils getStringSecurityProperty
INFO: Found string security property [jdk.tls.disabledAlgorithms]: SSLv3, RC4, MD5withRSA, DH keySize < 1024, EC keySize < 224, DES40_CBC, RC4_40, 3DES_EDE_CBC
Mar 16, 2021 11:55:45 PM org.bouncycastle.jsse.provider.PropertyUtils getStringSecurityProperty
INFO: Found string security property [jdk.certpath.disabledAlgorithms]: MD2, MD5, SHA1 jdkCA & usage TLSServer, RSA keySize < 1024, DSA keySize < 1024, EC keySize < 224
Mar 16, 2021 11:55:45 PM org.bouncycastle.jsse.provider.DisabledAlgorithmConstraints create
WARNING: Ignoring unsupported entry in 'jdk.certpath.disabledAlgorithms': SHA1 jdkCA & usage TLSServer
Mar 16, 2021 11:55:45 PM org.bouncycastle.jsse.provider.PropertyUtils getBooleanSecurityProperty
INFO: Found boolean security property [keystore.type.compat]: true
Mar 16, 2021 11:55:45 PM org.bouncycastle.jsse.provider.ProvTrustManagerFactorySpi getDefaultTrustStore
INFO: Initializing with trust store at path: /local/home/iggarish/jdk11/jdk-11/lib/security/cacerts
Exception in thread "main" java.sql.SQLException: Error loading the provided keystore.
at com.amazon.redshift.util.RedshiftException.getSQLException(RedshiftException.java:48)
at com.amazon.redshift.Driver.connect(Driver.java:303)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
at TestIAMUserConnectWithBouncyCastle.getIAMConnection(TestIAMUserConnectWithBouncyCastle.java:62)
at TestIAMUserConnectWithBouncyCastle.main(TestIAMUserConnectWithBouncyCastle.java:36)
Caused by: java.io.IOException: stream does not represent a PKCS12 key store
at org.bouncycastle.jcajce.provider.keystore.pkcs12.PKCS12KeyStoreSpi.engineLoad(Unknown Source)
at java.base/java.security.KeyStore.load(KeyStore.java:1479)
at com.amazon.redshift.ssl.LibPQFactory.getDefaultKeystore(LibPQFactory.java:376)
at com.amazon.redshift.ssl.LibPQFactory.getDefaultTrustManager(LibPQFactory.java:512)
at com.amazon.redshift.ssl.LibPQFactory.(LibPQFactory.java:148)
at com.amazon.redshift.core.SocketFactoryFactory.getSslSocketFactory(SocketFactoryFactory.java:63)
at com.amazon.redshift.ssl.MakeSSL.convert(MakeSSL.java:29)
at com.amazon.redshift.core.v3.ConnectionFactoryImpl.enableSSL(ConnectionFactoryImpl.java:487)

iggarish · 2021-03-17T00:16:08Z

ok. I fix the code as you suggest and it works fine now:
dev-dsk-iggarish-2a-77ebad06 % java TestIAMUserConnectWithBouncyCastle
BC
BCJSSE
SUN
SunRsaSign
SunEC
SunJSSE
SunJCE
SunJGSS
SunSASL
XMLDSig
SunPCSC
JdkLDAP
JdkSASL
SunPKCS11
type: pkcs12
Mar 17, 2021 12:15:09 AM org.bouncycastle.jsse.provider.PropertyUtils getStringSecurityProperty
INFO: Found string security property [jdk.tls.disabledAlgorithms]: SSLv3, RC4, MD5withRSA, DH keySize < 1024, EC keySize < 224, DES40_CBC, RC4_40, 3DES_EDE_CBC
Mar 17, 2021 12:15:09 AM org.bouncycastle.jsse.provider.PropertyUtils getStringSecurityProperty
INFO: Found string security property [jdk.certpath.disabledAlgorithms]: MD2, MD5, SHA1 jdkCA & usage TLSServer, RSA keySize < 1024, DSA keySize < 1024, EC keySize < 224
Mar 17, 2021 12:15:09 AM org.bouncycastle.jsse.provider.DisabledAlgorithmConstraints create
WARNING: Ignoring unsupported entry in 'jdk.certpath.disabledAlgorithms': SHA1 jdkCA & usage TLSServer
Mar 17, 2021 12:15:09 AM org.bouncycastle.jsse.provider.PropertyUtils getBooleanSecurityProperty
INFO: Found boolean security property [keystore.type.compat]: true
Mar 17, 2021 12:15:09 AM org.bouncycastle.jsse.provider.ProvTrustManagerFactorySpi getDefaultTrustStore
INFO: Initializing with trust store at path: /local/home/iggarish/jdk11/jdk-11/lib/security/cacerts
Connected... Driver Version: 2.0.0.0

iggarish · 2021-03-17T00:16:29Z

This fix will be available in next release plan for end of the month.

Syed-SnapLogic · 2021-03-17T05:13:31Z

This is great update, thank you so much @iggarish. May I please know the expected release date if its available?

Syed-SnapLogic · 2021-03-17T17:13:06Z

The release date would help us prepare our plan accordingly. So if you have had that pls do let us know.

iggarish · 2021-03-17T17:58:55Z

We are planning to release by end of this month. Exact date is not known, because it depends on many factors. But trying to release by 31-Mar. I will update here when actual release is ready.

Syed-SnapLogic · 2021-03-18T06:03:51Z

Thank you.

Syed-SnapLogic · 2021-03-24T12:44:53Z

Any update @iggarish ?

iggarish · 2021-03-24T16:13:03Z

31-Mar is still plan to release. I will update here when I actually release it.

Syed-SnapLogic · 2021-03-25T10:25:40Z

Thank you.

iggarish · 2021-03-28T20:15:52Z

Released 2.0.0.4 version. Check README file for download links.

iggarish closed this as completed Mar 28, 2021

sync-by-unito bot mentioned this issue Mar 29, 2021

Bump redshift-jdbc42 from 2.0.0.3 to 2.0.0.4 liquibase/liquibase-redshift#39

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Load Keystore Intelligently in the Driver #14

Load Keystore Intelligently in the Driver #14

Syed-SnapLogic commented Mar 15, 2021

iggarish commented Mar 15, 2021

iggarish commented Mar 16, 2021

Syed-SnapLogic commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 17, 2021

iggarish commented Mar 17, 2021

Syed-SnapLogic commented Mar 17, 2021

Syed-SnapLogic commented Mar 17, 2021

iggarish commented Mar 17, 2021

Syed-SnapLogic commented Mar 18, 2021

Syed-SnapLogic commented Mar 24, 2021

iggarish commented Mar 24, 2021

Syed-SnapLogic commented Mar 25, 2021

iggarish commented Mar 28, 2021

Load Keystore Intelligently in the Driver #14

Load Keystore Intelligently in the Driver #14

Comments

Syed-SnapLogic commented Mar 15, 2021

Driver version

Redshift version

Client Operating System

JAVA/JVM version

Table schema

Problem description

JDBC trace logs

Reproduction code

Recommendation

iggarish commented Mar 15, 2021

iggarish commented Mar 16, 2021

Syed-SnapLogic commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 16, 2021

iggarish commented Mar 17, 2021

iggarish commented Mar 17, 2021

Syed-SnapLogic commented Mar 17, 2021

Syed-SnapLogic commented Mar 17, 2021

iggarish commented Mar 17, 2021

Syed-SnapLogic commented Mar 18, 2021

Syed-SnapLogic commented Mar 24, 2021

iggarish commented Mar 24, 2021

Syed-SnapLogic commented Mar 25, 2021

iggarish commented Mar 28, 2021