Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shading tests #390

Merged
merged 5 commits into from
Jul 6, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,13 @@ All notable changes to this project will be documented in this file.
- Fixed leakage of Vibur and Tomcat JDBC test dependencies in `jdbc-test` and `mysql` modules (#382)
- Added timeout and retries for creation of `RemoteWebDriver` (#381, #373, #257)
- Fixed double encoding of listNetwork's filter until it's fixed in docker-java (#385)
- Fixed various shading issues

### Changed
- Added support for Docker networks (#372)
- Added `getFirstMappedPort` method
- Extracted Oracle XE container into a separate repository ([testcontainers/testcontainers-java-module-oracle-xe](https://github.com/testcontainers/testcontainers-java-module-oracle-xe))
- Added shading tests

## [1.3.1] - 2017-06-22
### Fixed
Expand Down
12 changes: 12 additions & 0 deletions core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@
<groupId>org.glassfish.jersey.connectors</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
Copy link
Member Author

@bsideup bsideup Jul 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a common rule not to ship logging implementation with libraries. I have no idea why @docker-java does that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>

Expand Down Expand Up @@ -228,9 +232,17 @@
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/NOTICE</exclude>
<exclude>META-INF/LICENSE</exclude>
<exclude>META-INF/DEPENDENCIES</exclude>
<exclude>META-INF/maven/</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>META-INF/io.netty.versions.properties</exclude>
<exclude>mozilla/public-suffix-list.txt</exclude>
<!-- README.md comes from docker-java -->
<exclude>META-INF/services/README.md</exclude>
<exclude>META-INF/services/com.fasterxml.jackson.core.*</exclude>
<exclude>META-INF/services/com.github.dockerjava.api.command.*</exclude>
<exclude>META-INF/services/javax.ws.rs.ext.*</exclude>
Expand Down
71 changes: 0 additions & 71 deletions modules/jdbc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,75 +26,4 @@
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"/>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
<relocations>
<relocation>
<pattern>com.google.common</pattern>
<shadedPattern>org.testcontainers.shaded.com.google.common</shadedPattern>
</relocation>
<relocation>
<pattern>org.glassfish</pattern>
<shadedPattern>org.testcontainers.shaded.org.glassfish</shadedPattern>
</relocation>
<relocation>
<pattern>javax.ws.rs</pattern>
<shadedPattern>org.testcontainers.shaded.javax.ws.rs</shadedPattern>
</relocation>
<relocation>
<pattern>com.fasterxml.jackson</pattern>
<shadedPattern>org.testcontainers.shaded.com.fasterxml.jackson</shadedPattern>
</relocation>
</relocations>
<filters>
<filter>
<artifact>org.bouncycastle:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/services/com.fasterxml.jackson.core.*</exclude>
<exclude>META-INF/services/com.github.dockerjava.api.command.*</exclude>
<exclude>META-INF/services/javax.ws.rs.ext.*</exclude>
<exclude>META-INF/services/org.glassfish.hk2.extension.*</exclude>
<exclude>META-INF/services/org.jvnet.hk2.external.generator.*</exclude>
<exclude>META-INF/services/org.glassfish.jersey.internal.spi.*</exclude>
</excludes>
</filter>
</filters>
<artifactSet>
<excludes>
<exclude>com.google.guava:*</exclude>
<exclude>org.glassfish.*:*</exclude>
</excludes>
</artifactSet>
<promoteTransitiveDependencies>true</promoteTransitiveDependencies>
<shadedArtifactAttached>false</shadedArtifactAttached>
</configuration>
</plugin>
</plugins>
</build>
</project>
5 changes: 0 additions & 5 deletions modules/mysql/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,6 @@
<name>TestContainers :: JDBC :: MySQL</name>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>testcontainers</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>jdbc</artifactId>
Expand Down
5 changes: 0 additions & 5 deletions modules/postgresql/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,6 @@
<name>TestContainers :: JDBC :: PostgreSQL</name>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>testcontainers</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>jdbc</artifactId>
Expand Down
54 changes: 0 additions & 54 deletions modules/selenium/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,6 @@
<artifactId>testcontainers</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.github.docker-java</groupId>
<artifactId>docker-java</artifactId>
<version>${docker-java.version}</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>18.0</version>
<scope>provided</scope>
</dependency>

<!-- WebDriver dependency as 'provided' scope -->
<dependency>
Expand All @@ -57,45 +44,4 @@
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<relocations>
<relocation>
<pattern>com.google.common</pattern>
<shadedPattern>org.testcontainers.shaded.com.google.common</shadedPattern>
</relocation>
</relocations>
<filters>
<filter>
<artifact>org.bouncycastle:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<artifactSet>
<excludes>
<exclude>com.google.guava:*</exclude>
</excludes>
</artifactSet>
<promoteTransitiveDependencies>true</promoteTransitiveDependencies>
<shadedArtifactAttached>false</shadedArtifactAttached>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
import java.util.Date;
import java.util.concurrent.TimeUnit;

import static com.google.common.base.Preconditions.checkState;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since it's the only place where we use Guava, I removed it from selenium module and inlined the implementation of checkState, so that we don't have to shade guava into the module

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 good idea - I thought it was more widely used but it's good to remove it!

import static java.time.temporal.ChronoUnit.SECONDS;

/**
Expand Down Expand Up @@ -87,8 +86,11 @@ protected Integer getLivenessCheckPort() {
@Override
protected void configure() {

checkState(desiredCapabilities != null);
if (! customImageNameIsSet) {
if (desiredCapabilities == null) {
throw new IllegalStateException();
}

if (!customImageNameIsSet) {
super.setDockerImageName(getImageForCapabilities(desiredCapabilities));
}

Expand Down
5 changes: 0 additions & 5 deletions modules/virtuoso/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,6 @@
</repositories>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>testcontainers</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>jdbc</artifactId>
Expand Down
5 changes: 5 additions & 0 deletions shade-test/clashing-deps-jackson/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@
<artifactId>clashing-deps-jackson</artifactId>

<dependencies>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
Expand Down
5 changes: 5 additions & 0 deletions shade-test/clashing-deps-jersey/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@
<artifactId>clashing-deps-jersey</artifactId>

<dependencies>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-client</artifactId>
Expand Down
24 changes: 24 additions & 0 deletions shade-test/jar-file/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>shade-test</artifactId>
<groupId>org.testcontainers</groupId>
<version>0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

<artifactId>jar-file</artifactId>

<dependencies>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.6.2</version>
<scope>test</scope>
</dependency>
</dependencies>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
package org.testcontainers;

import org.assertj.core.api.ListAssert;
import org.junit.*;

import java.io.IOException;
import java.net.URI;
import java.nio.file.*;

import static java.util.Collections.emptyMap;
import static org.assertj.core.api.Assertions.assertThat;

public class JarFileShadingTest {

private static FileSystem fileSystem;

private static Path root;

@BeforeClass
public static void setUp() throws Exception {
Path path = Paths.get("..", "..", "core", "target", "testcontainers-0-SNAPSHOT.jar");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is prone to refactor of locations.
How about you copy this jar to this project target directory using copy-dependencies goal of maven-dependencies-plugin? You can add it in the pom.xml of this project.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not very IDE-friendly I'm afraid.

Test will fail if we refactor the locations => should be fine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please elaborate why is it not IDE friendly?
I though one of Maven projects best practice is to avoid depending on explicit locations, but depends on artifacts. When you use copy dependencies you alleviate your self from that since you use GAV and not relative paths. You are free then to place core where ever you want.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We run tests from IDE, and would like to keep that. Doing something in Maven means that we have to run the tests with Maven and only Maven.

Personally, I would prefer to use Gradle and "delegate build to Gradle" feature of IntelliJ, but we use Maven ATM.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this test you rely on the fact that someone ran mvn install on the core project right? Once he has done that you can the test from the IDE.
I'm saying the same thing - someone must run mvn install on the core project and on the project it self. Then you run the test from the IDE - same.
So both rely on manual mvn operation.
Am I missing something here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then how does it matter where the jar file is? Again, the test will fail if we change paths.

Doing "copy dependency" might lead to the dependency copied from Maven's local repo => more magical and error-prone

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's going to be pretty obvious why this has broken if someone ever manages to break it, so I think we'll be OK with this.


fileSystem = FileSystems.newFileSystem(URI.create("jar:" + path.toUri()), emptyMap());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool trick!


root = fileSystem.getRootDirectories().iterator().next();
}

@AfterClass
public static void tearDown() throws Exception {
fileSystem.close();
}

@Test
public void testPackages() throws Exception {
assertThatFileList(root).containsOnly(
"docker-java.properties",
"org/",
"META-INF/",
"com/"
);

assertThatFileList(root.resolve("org")).containsOnly(
"testcontainers/"
);

assertThatFileList(root.resolve("com")).containsOnly(
"github/"
);

assertThatFileList(root.resolve("com").resolve("github")).containsOnly(
"dockerjava/"
);
}

@Test
public void testMetaInf() throws Exception {
assertThatFileList(root.resolve("META-INF")).containsOnly(
"MANIFEST.MF",
"services/",
"native/"
);

assertThatFileList(root.resolve("META-INF").resolve("native")).containsOnly(
"liborg-testcontainers-shaded-netty-transport-native-epoll.so"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funny that we shade something which is architecture specific - I mean this might not work for Mac or Windows?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not used where not supposed to be :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? Docker Java Client doesn't use Netty communicate over TCP to Docker daemon? Does it do it using this native library or plain nio?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We choose different strategies on different platforms, also Netty has a mechanism where they optionally load epoll library where it's supported ( Linux )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing that this will have to change when merged with #393 and netty-transport-native-kqueue comes in. NBD though!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rnorth yeah, I'll rebase #393 after we merge this one :)

);
}

@Test
public void testMetaInfServices() throws Exception {
assertThatFileList(root.resolve("META-INF").resolve("services"))
.allMatch(it -> it.startsWith("org.testcontainers."));
}

private ListAssert<String> assertThatFileList(Path path) throws IOException {
return (ListAssert) assertThat(Files.list(path))
.extracting(Path::getFileName)
.extracting(Object::toString);
}
}
14 changes: 1 addition & 13 deletions shade-test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,6 @@
<module>clashing-deps-jackson</module>
<module>clashing-deps-jersey</module>
<module>service-lookup-dropwizard</module>
<module>jar-file</module>
</modules>

<dependencies>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.rnorth.visible-assertions</groupId>
<artifactId>visible-assertions</artifactId>
<version>1.0.5</version>
</dependency>
</dependencies>
</project>
Loading