UnixPB: Include Pre-Req Packages For SSL Test (#3169)

* Include SSL Packages

Include pre-requisite packages for running SSL client tests.

* Add VPC SSL tests for Linux

* Fix missing bracket.

* Revert debian URL change

* Add twisty section

* Fix twisty section

* Update ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Alpine.yml

Co-authored-by: Stewart X Addison <6487691+sxa@users.noreply.github.com>

* Update DockerStatic Files For SSL Tests

* Fix errant comment.

* Seperate ssl packages for alpine for consistency

---------

Co-authored-by: Stewart X Addison <6487691+sxa@users.noreply.github.com>

FAQ.md

@@ -211,6 +211,66 @@ should have an underscore `_` prepended to it.
 (For the last one, that makes use of the system.custom target added via
 [this PR](https://github.com/AdoptOpenJDK/openjdk-tests/pull/2234))
 
+## Running The SSL Test Suites
+<details>
+<summary>Quick Guide To Running The SSL Test Suites</summary>
+
+As part of the fix for infrastructure [issue 3059](https://github.com/adoptium/infrastructure/issues/3059) several new pre-requisite packages have been added to the Unix playbooks, usually things such as (gnutls, gnutls-utils, libnss3.so, libnssutil3.so, nss-devel, nss-tools) or their O/S specific variants. In order to validate that these tests can run following any changes, the following process can be followed once the playbooks have been run successfully:
+
+N.B. Currently the integration testing for other clients is currently not enabed on non-Linux platforms.
+
+1) Clone The Open JDK ssl test suites
+
+```
+git clone https://github.com/rh-openjdk/ssl-tests
+
+```
+
+2) Download and install the JDK to be tested, and export the TESTJAVA environment variable.
+```
+export TESTJAVA=/home/user/jdk17
+```
+
+3) Execute The 3 Test Suites To Test External clients, from the directory the git clone of the openjdk ssl test suites was carried out:
+```
+cd ssl-tests/jtreg-wrappers
+
+Run each of the following test suites:
+
+./ssl-tests-gnutls-client.sh
+./ssl-tests-nss-client.sh
+./ssl-tests-openssl-client.sh
+```
+
+Each script should produce output similar to the below, with some tests being completed, and others skipped, but as long as the tests run without errors, this can be considered a success.
+
+```
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_RSA_WITH_AES_256_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_DSS_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_RSA_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_DSS_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_256_GCM_SHA384
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_128_GCM_SHA256
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_256_CBC_SHA256
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_128_CBC_SHA256
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_EMPTY_RENEGOTIATION_INFO_SCSV
+
+```
+
+N.B. Due to a missing pre-requisite binary(tstclnt) not being available in the nss packages on Alpine, OpenSuse or SLES, the ssl-tests-nss-client.sh tests can not be run.
+
+</details>
+
 ## Testing changes
 
 If you are making a change which might have a negative effect on the
@@ -251,7 +311,7 @@ the Adoptium projects, and people can be given "contributor" or
 [the wiki](https://github.com/adoptium/adoptium/wiki/Working-with-Eclipse) for
 the processes around this) to the repositories which are under each Adoptium
 project as per
-[this comment](https://github.com/adoptium/infrastructure/issues/2549#issuecomment-1178903957). 
+[this comment](https://github.com/adoptium/infrastructure/issues/2549#issuecomment-1178903957).
 Most of the relevant ones are under the
 [temurin](https://projects.eclipse.org/projects/adoptium.temurin/who)
 or [aqavit](https://projects.eclipse.org/projects/adoptium.aqavit) projects.

ansible/pbTestScripts/testJDK.sh

@@ -36,3 +36,26 @@ else
 	$MAKE_COMMAND compile
 	$MAKE_COMMAND _MBCS_Tests_pref_ja_JP_linux_0
 fi
+
+# Run SSL Client Tests Linux Only ( Not Solaris / FreeBSD )
+if [[ "$(uname)" == "FreeBSD" ]] || [["$(uname)" == "SunOS"]]; then
+	echo "Skipping SSL Tests As Not Supported"
+else
+	export TESTJAVA=$TEST_JDK_HOME
+	echo DEBUG: TESTJAVA = $TEST_JDK_HOME
+	mkdir -p $HOME/testLocation
+  [ ! -d $HOME/testLocation/ssl-tests ] && git clone https://github.com/rh-openjdk/ssl-tests $HOME/testLocation/ssl-tests
+	cd $HOME/testLocation/ssl-tests/jtreg-wrappers
+	ls -l
+	# Reduce Tests For Alpine/Sles/OpenSuse
+	if [[ "$(uname -v)" =~ .*"Alpine"*. ]] || [[ `cat /etc/os-release|grep -i opensuse|wc -l` -gt 0 ]] || [[ `cat /etc/os-release|grep -i SLES|wc -l` -gt 0 ]] ; then
+		echo "Run Alpine/OpenSuse/Sles SSL Client Tests"
+		./ssl-tests-gnutls-client.sh
+		./ssl-tests-openssl-client.sh
+	else
+		echo "Run Full Set Of SSL Client Tests"
+		./ssl-tests-gnutls-client.sh
+		./ssl-tests-nss-client.sh
+		./ssl-tests-openssl-client.sh
+  fi
+fi

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/tasks/Solaris.yml

@@ -443,7 +443,7 @@
     - not liberica11_installed.stat.exists
   tags: build_tools
 
-- name: Check for /usr/lib/jbm/fallocate.so
+- name: Check for /usr/lib/jvm/fallocate.so
   stat:
     path: /usr/lib/jvm/fallocate.so
   register: fallocate_installed

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Alpine.yml

@@ -20,6 +20,9 @@ Build_Tool_Packages:
   - freetype-dev
   - grep
   - gnupg
+  - gnutls                        # OpenSSL tests
+  - gnutls-dev                    # OpenSSL tests
+  - gnutls-utils                  # OpenSSL tests
   - libdwarf                      # OpenJ9
   - libdwarf-dev                  # OpenJ9
   - libx11
@@ -35,6 +38,9 @@ Build_Tool_Packages:
   - libxtst
   - libxtst-dev
   - linux-headers
+  - nss
+  - nss-dev
+  - nss-tools
   - numactl
   - numactl-dev                   # OpenJ9
   - pigz                          # Used in preference to gzip for tar.gz'ing
@@ -59,3 +65,9 @@ Test_Tool_Packages:
   - xauth
   - xorg-server
   - xvfb
+  - gnutls
+  - gnutls-dev
+  - gnutls-utils
+  - nss
+  - nss-dev
+  - nss-tools

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/CentOS.yml

@@ -26,8 +26,12 @@ Build_Tool_Packages:
   - glibc-common
   - glibc-devel
   - gmp-devel
+  - gnutls
+  - gnutls-utils
   - java-1.8.0-openjdk-devel
   - libcurl-devel
+  - libnss3.so
+  - libnssutil3.so
   - libpng-devel
   - libXext-devel
   - libXi-devel                   # JDK12+ compilation
@@ -38,6 +42,8 @@ Build_Tool_Packages:
   - make
   - mesa-libGL-devel
   - mpfr-devel
+  - nss-devel
+  - nss-tools
   - numactl-devel                 # OpenJ9
   - openssh-clients               # IBM: cloning over SSH
   - openssl-devel
@@ -105,3 +111,9 @@ Test_Tool_Packages:
   - xorg-x11-xauth
   - xorg-x11-server-Xvfb
   - fakeroot
+  - gnutls
+  - gnutls-utils
+  - libnss3.so
+  - libnssutil3.so
+  - nss-devel
+  - nss-tools

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Debian.yml

@@ -16,6 +16,7 @@ Build_Tool_Packages:
   - g++
   - gcc
   - gettext
+  - gnutls-bin
   - libasound2-dev
   - libcapstone-dev
   - libcups2-dev
@@ -29,6 +30,9 @@ Build_Tool_Packages:
   - libgmp3-dev
   - libmpfr-dev
   - libmpfr-doc
+  - libnss3
+  - libnss3-dev
+  - libnss3-tools
   - libssl-dev
   - libwww-perl
   - libx11-dev
@@ -98,6 +102,10 @@ Test_Tool_Packages:
   - xvfb
   - binfmt-support
   - qemu-user-static
+  - gnutls-bin
+  - libnss3
+  - libnss3-dev
+  - libnss3-tools
 
 Test_Tool_Packages_x86_64:
   - pulseaudio

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Fedora.yml

@@ -27,8 +27,11 @@ Build_Tool_Packages:
   - glibc-common
   - glibc-devel
   - gmp-devel
+  - gnutls
+  - gnutls-utils
   - libcurl-devel
   - libffi-devel
+  - libnss3.so
   - libpng-devel
   - libXext-devel
   - libXi-devel                   # JDK12+ compilation
@@ -39,6 +42,8 @@ Build_Tool_Packages:
   - make
   - mesa-libGL-devel
   - mpfr-devel
+  - nss-devel
+  - nss-tools
   - openssl-devel
   - perl-devel
   - pkgconfig
@@ -128,3 +133,8 @@ Test_Tool_Packages:
   - expat-devel
   - libcurl-devel
   - mercurial
+  - gnutls
+  - gnutls-utils
+  - libnss3.so
+  - nss-devel
+  - nss-tools

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/RedHat.yml

@@ -20,6 +20,8 @@ Build_Tool_Packages:
   - fontconfig-devel
   - freetype-devel
   - gnupg
+  - gnutls
+  - gnutls-utils
   - gcc
   - gcc-c++
   - gettext
@@ -29,6 +31,7 @@ Build_Tool_Packages:
   - gmp-devel
   - libcurl-devel
   - libffi-devel
+  - libnss3.so
   - libpng-devel
   - libXext-devel
   - libXi-devel                   # JDK12+ compilation
@@ -39,6 +42,8 @@ Build_Tool_Packages:
   - make
   - mesa-libGL-devel
   - mpfr-devel
+  - nss-devel
+  - nss-tools
   - openssl-devel
   - perl-devel
   - perl-IPC-Cmd                  # required for openssl v3 compiles
@@ -107,3 +112,8 @@ Test_Tool_Packages:
   - expat-devel
   - libcurl-devel
   - mercurial
+  - gnutls
+  - gnutls-utils
+  - libnss3.so
+  - nss-devel
+  - nss-tools

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/SLES.yml

@@ -16,9 +16,13 @@ Build_Tool_Packages:
   - gcc-c++
   - glibc
   - glibc-devel
+  - gnutls
   - libdw1
   - libelf1
   - make
+  - mozilla-nss
+  - mozilla-nss-devel
+  - mozilla-nss-tools
   - pkg-config
   - unzip
   - wget
@@ -92,3 +96,7 @@ Test_Tool_Packages:
   - xorg-x11-server
   - xorg-x11-server-extra
   - glibc-locale                  # Internationalization tests
+  - gnutls
+  - mozilla-nss
+  - mozilla-nss-devel
+  - mozilla-nss-tools

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Ubuntu.yml

@@ -18,6 +18,7 @@ Build_Tool_Packages:
   - gettext
   - git
   - gnupg
+  - gnutls-bin
   - libasound2-dev
   - libcapstone-dev
   - libcups2-dev
@@ -31,6 +32,9 @@ Build_Tool_Packages:
   - libgmp3-dev
   - libmpfr-dev
   - libmpfr-doc
+  - libnss3
+  - libnss3-tools
+  - libnss3-dev
   - libssl-dev
   - libwww-perl
   - libx11-dev
@@ -106,6 +110,10 @@ Test_Tool_Packages:
   - libexpat1-dev
   - libcurl4-openssl-dev
   - fakeroot
+  - gnutls-bin
+  - libnss3
+  - libnss3-tools
+  - libnss3-dev
 
 Test_Tool_Packages_x86_64:
   - pulseaudio

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/openSUSE.yml

@@ -18,12 +18,14 @@ Build_Tool_Packages:
   - gcc-c++
   - glibc
   - glibc-devel
+  - gnutls
   - libcurl-devel
   - libdw1
   - libdwarf-devel
   - libelf-devel
   - libelf0
   - libelf1
+  - libnss3.so
   - libnuma-devel
   - libpng15-devel
   - libXext-devel
@@ -33,6 +35,7 @@ Build_Tool_Packages:
   - libXt-devel
   - libXtst-devel
   - make
+  - mozilla-nss
   - ntp
   - numactl
   - pkg-config
@@ -63,3 +66,6 @@ Test_Tool_Packages:
   - xorg-x11
   - xorg-x11-devel
   - glibc-locale                  # Internationalization tests
+  - gnutls
+  - libnss3.so
+  - mozilla-nss

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp311

@@ -2,6 +2,10 @@ FROM alpine:3.11
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
         libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp312

@@ -2,6 +2,10 @@ FROM alpine:3.12
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
         libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp313

@@ -2,6 +2,10 @@ FROM alpine:3.13
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
         libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp314

@@ -2,6 +2,10 @@ FROM alpine:3.14
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
         libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)