Import Spring Boot's dependency BOM, fix spring-boot:run at parent pr…

…oject level (#276) * Fix using spring-boot:run from project root A little fix when some subproject modules are Spring Boot apps, but the parent is not. Also document how to overcome IntelliJ CE woes with Spring Boot, and inter-module Maven dependencies. * Get Spring managed dep versions via their BOM Spring has done the work of finding direct and transitive versions of dependencies compatible with their ecosystem, and declaring them as a BOM. It's in our best interest to heed it as much as we can. * ci: Try --also-make instead of skipping Enforcer Per discussion: #271 (comment) The `install` step should not be necessary for the unit tests, but we should use `--batch-mode` on the test step for CI. * Remove maven-shade-plugin from core It was resulting in a non-functional JAR, appears to be incomplete configuration. It seems unlikely to me that core will be used as a library, if there is a need the shading configuration can be fixed but it probably needs other considerations anyway.
feast-dev · Oct 30, 2019 · 18d54b7 · 18d54b7
1 parent 9cc1673
commit 18d54b7
Show file tree

Hide file tree

Showing 7 changed files with 124 additions and 191 deletions.
diff --git a/.prow/scripts/test-serving.sh b/.prow/scripts/test-serving.sh
@@ -4,14 +4,11 @@
     --archive-uri gs://feast-templocation-kf-feast/.m2.2019-10-24.tar \
     --output-dir /root/
 
-# Skip Maven enforcer: https://stackoverflow.com/questions/50647223/maven-enforcer-issue-when-running-from-reactor-level
-mvn --projects serving --batch-mode --define skipTests=true \
-    --define enforcer.skip=true clean install
-mvn --projects serving --define enforcer.skip=true test
+mvn --batch-mode --also-make --projects serving test
 TEST_EXIT_CODE=$?
 
 # Default artifact location setting in Prow jobs
 LOGS_ARTIFACT_PATH=/logs/artifacts
 cp -r serving/target/surefire-reports ${LOGS_ARTIFACT_PATH}/surefire-reports
 
-exit ${TEST_EXIT_CODE}
+exit ${TEST_EXIT_CODE}
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -223,7 +223,55 @@ entity_dataset {
 '
 ```
 
-#### Tips for quickly running Postgres, Redis and Kafka locally with Docker
+## Development
+
+Notes:
+
+  - Use of Lombok is being phased out, prefer to use [Google Auto] in new code.
+
+[Google Auto]: https://github.com/google/auto
+
+### Running Unit Tests
+
+    $ mvn test
+
+### Running Integration Tests
+
+_Note: integration suite isn't yet separated from unit._
+
+    $ mvn verify
+
+### Running Components Locally
+
+The `core` and `serving` modules are Spring Boot applications. These may be run as usual for [the Spring Boot Maven plugin][boot-maven]:
+
+    $ mvn --also-make --projects core sprint-boot:run
+
+    # Or for short:
+    $ mvn -am -pl core spring-boot:run
+
+Note the use of `--also-make` since some components depend on library modules from within the project.
+
+[boot-maven]: https://docs.spring.io/spring-boot/docs/current/maven-plugin/index.html
+
+#### Running From IntelliJ
+
+IntelliJ IDEA Ultimate has built-in support for Spring Boot projects, so everything may work out of the box. The Community Edition needs help with two matters:
+
+1. The IDE is [not clever enough][idea-also-make] to apply `--also-make` for Maven when it should.
+1. The Spring Boot Maven plugin automatically puts dependencies with `provided` scope on the runtime classpath when using `spring-boot:run`, such as its embedded Tomcat server. The "Play" buttons in the gutter or right-click menu of a `main()` method [do not do this][idea-boot-main].
+
+Fortunately there is one simple way to address both:
+
+1. Open `View > Tool Windows > Maven`
+1. Drill down to e.g. `Feast Core > Plugins > spring-boot:run`, right-click and `Create 'feast-core [spring-boot'…`
+1. In the dialog that pops up, check the `Resolve Workspace artifacts` box
+1. Click `OK`. You should now be able to select this run configuration for the Play button in the main toolbar, keyboard shortcuts, etc.
+
+[idea-also-make]: https://stackoverflow.com/questions/15073877/using-mavens-also-make-option-in-intellij
+[idea-boot-main]: https://stackoverflow.com/questions/30237768/run-spring-boots-main-using-ide
+
+#### Tips for Running Postgres, Redis and Kafka with Docker
 
 This guide assumes you are running Docker service on a bridge network (which
 is usually the case if you're running Linux). Otherwise, you may need to

diff --git a/core/pom.xml b/core/pom.xml
@@ -33,45 +33,11 @@
     <build>
         <plugins>
             <plugin>
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-shade-plugin</artifactId>
-                <version>2.4.3</version>
+                <groupId>org.springframework.boot</groupId>
+                <artifactId>spring-boot-maven-plugin</artifactId>
                 <configuration>
-                    <transformers>
-                        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
-                            <resource>META-INF/spring.handlers</resource>
-                        </transformer>
-                        <transformer implementation="org.springframework.boot.maven.PropertiesMergingResourceTransformer">
-                            <resource>META-INF/spring.factories</resource>
-                        </transformer>
-                        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
-                            <resource>META-INF/spring.schemas</resource>
-                        </transformer>
-                        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
-                            <resource>META-INF/spring.components</resource>
-                        </transformer>
-                        <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
-                            <mainClass>feast.core.CoreApplication</mainClass>
-                        </transformer>
-                        <transformer implementation="org.apache.maven.plugins.shade.resource.ComponentsXmlResourceTransformer" />
-                        <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
-                    </transformers>
+                    <skip>false</skip>
                 </configuration>
-                <executions>
-                    <execution>
-                        <phase>package</phase>
-                        <goals>
-                            <goal>shade</goal>
-                        </goals>
-                    </execution>
-                </executions>
-                <dependencies>
-                    <dependency>
-                        <groupId>org.springframework.boot</groupId>
-                        <artifactId>spring-boot-maven-plugin</artifactId>
-                        <version>${springBootVersion}</version>
-                    </dependency>
-                </dependencies>
             </plugin>
             <plugin>
                 <groupId>org.xolstice.maven.plugins</groupId>
@@ -87,6 +53,14 @@
             <version>${project.version}</version>
         </dependency>
 
+        <!-- Hot reloading for Spring Boot. spring-boot-maven-plugin removes
+             this automatically when packaging. -->
+        <dependency>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-devtools</artifactId>
+            <optional>true</optional>
+        </dependency>
+
         <dependency>
           <groupId>javax.inject</groupId>
           <artifactId>javax.inject</artifactId>
@@ -111,7 +85,6 @@
         <dependency>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-data-jpa</artifactId>
-            <version>${springBootVersion}</version>
         </dependency>
         <!--compile "org.springframework.boot:spring-boot-starter-actuator:${springBootVersion}"-->
         <dependency>
@@ -125,7 +98,6 @@
             <artifactId>spring-boot-configuration-processor</artifactId>
         </dependency>
 
-
         <!--compile "io.grpc:grpc-services:${grpcVersion}"-->
         <dependency>
             <groupId>io.grpc</groupId>
@@ -175,8 +147,8 @@
         <dependency>
             <groupId>org.postgresql</groupId>
             <artifactId>postgresql</artifactId>
-            <version>42.2.5</version>
-            <scope>runtime</scope>
+            <scope>provided</scope>
+            <optional>true</optional>
         </dependency>
 
         <dependency>
@@ -190,10 +162,9 @@
             <artifactId>lombok</artifactId>
         </dependency>
 
-        <!--testCompile 'org.hamcrest:hamcrest-all:1.3'-->
         <dependency>
             <groupId>org.hamcrest</groupId>
-            <artifactId>hamcrest-all</artifactId>
+            <artifactId>hamcrest-library</artifactId>
         </dependency>
 
         <!--testCompile 'com.jayway.jsonpath:json-path-assert:2.2.0'-->
@@ -213,10 +184,12 @@
         <dependency>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-test</artifactId>
+            <scope>test</scope>
         </dependency>
         <dependency>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-test-autoconfigure</artifactId>
+            <scope>test</scope>
         </dependency>
     </dependencies>
 </project>
diff --git a/core/src/test/java/feast/core/job/dataflow/DataflowJobMonitorTest.java b/core/src/test/java/feast/core/job/dataflow/DataflowJobMonitorTest.java
@@ -35,7 +35,6 @@
 import feast.types.FieldProto.Field;
 import feast.types.ValueProto.BoolList;
 import feast.types.ValueProto.Value;
-import feast.types.ValueProto.ValueType;
 import java.io.IOException;
 import org.junit.Before;
 import org.junit.Test;

diff --git a/ingestion/pom.xml b/ingestion/pom.xml
@@ -248,7 +248,6 @@
     <dependency>
       <groupId>org.apache.kafka</groupId>
       <artifactId>kafka_2.12</artifactId>
-      <version>2.3.0</version>
       <scope>test</scope>
     </dependency>