One app to rule them all

.github/workflows/build-1.x.yml

@@ -17,7 +17,7 @@ jobs:
       - uses: actions/setup-java@v2
         with:
           distribution: 'adopt'
-          java-version: '8'
+          java-version: '11'
           check-latest: true
       - name: Gradle Build
         run: |

.gitignore

@@ -12,3 +12,4 @@ build
 bin/
 .bloop/
 .metals/
+islandora-alpaca-app/src/main/resources/alpaca.properties

README.md

@@ -10,11 +10,215 @@ Event-driven middleware based on [Apache Camel](http://camel.apache.org/) that s
 
 ## Requirements
 
-This project requires Java 8 and can be built with [Gradle](https://gradle.org). To build and test locally, use `./gradlew build`.
+This project requires Java 11 and can be built with [Gradle](https://gradle.org).
+
+To build and test locally, clone this repository and then change into the Alpaca directory.
+Next run `./gradlew clean build shadowJar`.
+
+The main executable jar is available in the `islandora-alpaca-app/build/libs` directory, with the classifier `-all`.
+
+ie.
+> islandora-alpaca-app/build/libs/islandora-alpaca-app-2.0.0-all.jar
+
+## Configuration
+
+Alpaca is made up of several services, each of these can be enabled or disabled individually.
+
+Alpaca takes an external file to configure its behaviour.
+
+Look at the [`example.properties`](example.properties) file to see some example settings.
+
+The properties are:
+
+```
+# Common options
+error.maxRedeliveries=4
+```
+This defines how many times to retry a message before failing completely.
+
+There are also common ActiveMQ properties to setup the connection.
+```
+# ActiveMQ options
+jms.brokerUrl=tcp://localhost:61616
+```
+This defines the url to the ActiveMQ broker.
+```
+jms.username=
+jms.password=
+```
+This defines the login credentials (if required)
+```
+jms.connections=10
+```
+This defines the pool of connections to the ActiveMQ instance.
+```
+jms.concurrent-consumers=1
+```
+This defines how many messages to process simultaneously.
+
+### islandora-indexing-fcrepo
+
+This service manages a Drupal node into a corresponding Fedora resource.
+
+It's properties are:
+
+```
+# Fcrepo indexer options
+fcrepo.indexer.enabled=true
+```
+This defines whether the Fedora indexer is enabled or not.
+```
+fcrepo.indexer.node=queue:islandora-indexing-fcrepo-content
+fcrepo.indexer.delete=queue:islandora-indexing-fcrepo-delete
+fcrepo.indexer.media=queue:islandora-indexing-fcrepo-media
+fcrepo.indexer.external=queue:islandora-indexing-fcrepo-file-external
+```
+These define the various queues to listen on for the indexing/deletion
+messages. The part after `queue:` should match your Islandora instance "Actions".
+```
+fcrepo.indexer.milliner.baseUrl=http://localhost:8000/milliner
+```
+This defines the location of your Milliner microservice.
+```
+fcrepo.indexer.concurrent-consumers=1
+fcrepo.indexer.max-concurrent-consumers=1
+```
+These define the default number of concurrent consumers and maximum number of concurrent
+consumers working off your ActiveMQ instance.
+A value of `-1` means no setting is applied.
+
+### islandora-indexing-triplestore
+
+This service indexes the Drupal node into the configured triplestore
+
+It's properties are:
+
+```
+# Triplestore indexer options
+triplestore.indexer.enabled=false
+```
+This defines whether the Triplestore indexer is enabled or not.
+```
+triplestore.index.stream=queue:islandora-indexing-triplestore-index
+triplestore.delete.stream=queue:islandora-indexing-triplestore-delete
+```
+These define the various queues to listen on for the indexing/deletion
+messages. The part after `queue:` should match your Islandora instance "Actions".
+```
+triplestore.baseUrl=http://localhost:8080/bigdata/namespace/kb/sparql
+```
+This defines the location of your triplestore's SPARQL update endpoint.
+```
+triplestore.indexer.concurrent-consumers=1
+triplestore.indexer.max-concurrent-consumers=1
+```
+These define the default number of concurrent consumers and maximum number of concurrent
+consumers working off your ActiveMQ instance.
+A value of `-1` means no setting is applied.
+
+### islandora-connector-derivative
+
+This service is used to configure an external microservice. This service will deploy multiple copies of its routes
+with different configured inputs and outputs based on properties.
+
+The routes to be configured are defined with the property `derivative.systems.installed` which expects
+a comma separated list. Each item in the list defines a new route and must also define 3 additional properties.
+
+```
+derivative.<item>.enabled=true
+```
+This defines if the `item` service is enabled.
+```
+derivative.<item>.in.stream=queue:islandora-item-connector.index
+```
+This is the input queue for the derivative microservice.
+The part after `queue:` should match your Islandora instance "Actions".
+```
+derivative.<item>.service.url=http://example.org/derivative/convert
+```
+This is the microservice URL to process the request.
+```
+derivative.<item>.concurrent-consumers=1
+derivative.<item>.max-concurrent-consumers=1
+```
+These define the default number of concurrent consumers and maximum number of concurrent
+consumers working off your ActiveMQ instance.
+A value of `-1` means no setting is applied.
+
+For example, with two services defined (houdini and crayfits) my configuration would have
+```
+derivative.systems.installed=houdini,fits
+
+derivative.houdini.enabled=true
+derivative.houdini.in.stream=queue:islandora-connector-houdini
+derivative.houdini.service.url=http://127.0.0.1:8000/houdini/convert
+derivative.houdini.concurrent-consumers=1
+derivative.houdini.max-concurrent-consumers=4
+
+derivative.fits.enabled=true
+derivative.fits.in.stream=queue:islandora-connector-fits
+derivative.fits.service.url=http://127.0.0.1:8000/crayfits
+derivative.fits.concurrent-consumers=2
+derivative.fits.max-concurrent-consumers=2
+```
+
+### Customizing HTTP client timeouts
+
+You can alter the HTTP client from the defaults for its request, connection and socket timeouts.
+To do this you want to enable the request configurer.
+```shell
+request.configurer.enabled=true
+```
+
+Then set the next 3 timeouts (measured in milliseconds) to the desired timeout.
+```shell
+request.timeout=-1
+connection.timeout=-1
+socket.timeout=-1
+```
+
+The default for all three is `-1` which indicates no timeout.
+
+## Deploying/Running
+
+You can see the options by passing the `-h|--help` flag
+```shell
+> java -jar  islandora-alpaca-app/build/libs/islandora-alpaca-app-2.0.0-all.jar -h
+Usage: alpaca [-hV] [-c=<configurationFilePath>]
+  -h, --help      Show this help message and exit.
+  -V, --version   Print version information and exit.
+  -c, --config=<configurationFilePath>
+                  The path to the configuration file
+```
+
+Using the `-V|--version` flag will just return the current version of the application.
+```shell
+> java -jar  islandora-alpaca-app/build/libs/islandora-alpaca-app-2.0.0-all.jar -v
+2.0.0
+```
+
+To start Alpaca you would pass the external property file with the `-c|--config` flag.
+
+For example if you are using an external properties file located at `/opt/my.properties`,
+you would run:
+
+```shell
+java -jar islandora-alpaca-app-2.0.0-all.jar -c /opt/my.properties
+```
+
+## Debugging/Troubleshooting
+
+Logging is done to the console, and defaults to the INFO level. To get more verbose logging you
+can use the Java property `islandora.alpaca.log`
+
+i.e.
+```shell
+java -Dislandora.alpaca.log=DEBUG -jar islandora-alpaca-app-2.0.0-all.jar -c /opt/my.properties
+```
 
 ## Documentation
 
-Further documentation for this module is available on the [Islandora 8 documentation site](https://islandora.github.io/documentation/).
+Further documentation for this module is available on the [Islandora documentation site](https://islandora.github.io/documentation/).
 
 ## Troubleshooting/Issues