Fix DB unit tests

[Kehrlann]

Context

@fhanik and I have noticed that some unit tests did not respect the profile that was passed in, and always ran against an HSQL database, or were ignored

When you ran:

./gradlew '-Dspring.profiles.active=postgresql' test

without a postgres database running, the tests would pass, which should not be the case when a database is involved.

This PR only addresses tests, by fixing this issue, fixing failing tests and adding tooling.

Fixes

The tests in question:

• Any test with @WithDatabaseContext ; because that annotation had @ActiveProfiles("default"). We removed it.
• TableAndColumnNormalizationTest, again annotated with @ActiveProfiles("default")
• And possibly some, but not all, the tests annotated with @DefaultTestContext

When we re-enabled the multi-profile support, we had issues in some tests, notably with Postgres. We have fixed those issues.

Addition: scripts/docker-compose.yml

We added a docker compose file to have MySQL and Postgres running locally. Using run-unit-tests.sh was not working well on our ARM-based macs. This is not expected to be the final setup, but a first step to help us run tests fast.

Addition: @EnabledWithProfile / @DisabledWithProfile

To selectively turn off some tests based on profiles ; instead of relying on custom use of Assume.assumeXXX and Assumptions.assumeXXX which are harder to locate in the code.

See the javadoc for their usage.

Addition: @WithTimeZone

MySQL + the JDBC driver have issues with default time zones. We added an extension to hardcode the timezone to UTC in some unit tests, similar to what JUnit Pioneer does.

[strehle]

please check if https://github.com/cloudfoundry/uaa/blob/develop/scripts/start_db_helper.sh can be re-used

[fhanik]

PR still in Work in Progress mode

We wrote this on MacOS, I see failures on Linux. Working on those today

[fhanik]

New findings

1. Tests have database connection leaks. Many tests use JdbcTemplate.getDataSource().getConnection() and never call connection.close()

2. The @WithTimeZone cause some tests to fail. Need to be investigated. For now, this annotation is limited to mysql

3. Columns have mismatched data types causing failure in later versions of MySQL (users.id is a padded char while authz_approvals.user_id is a varchar.

4. Because tests didn't run against MySQL or PostgreSQL, we found many places where the word groups was used in queries and cause BadSqlGrammarException

5. All the MockMvc tests also had hardcoded to use in memory HSQLDB. This was fixed.

6. MySQL has in later versions deprecated PAD_CHAR_TO_FULL_LENGTH so we have converted users.id and oauth_client_details.created_by to VARCHAR

Items Left to Do

1. We can add maildev to docker-compose.yml (not to conflict with the in memory SMTP server SimpleSmtpServer that the integration tests run
2. We should enhance the LDAP image in docker-compose.yml to properly configure and seed the LDAP server - see scripts/integration-tests.sh
3. RateLimiter for cargo is causing 429 error on my Macbook M4
4. /etc/hosts requires the following entrie

127.0.0.1 testzone1.localhost
127.0.0.1 testzone2.localhost
127.0.0.1 testzone3.localhost
127.0.0.1 testzone4.localhost
127.0.0.1 testzonedoesnotexist.localhost
127.0.0.1 oidcloginit.localhost
127.0.0.1 testzoneinactive.localhost

5. Verify that hsqldb isn't unintentionally used in any tests

6. Fix remaining integration tests - current status has IT failures (assuming rate limits have been adjusted) - and one of them relates to LDAP preloading data (bullet 2 above) LdapIntegrationTests.test_LDAP_Custom_User_Attributes_In_ID_Token

7. is CHAR to VARCHAR a breaking change?