Use Apache Jena Fuseki docker image #1477
Conversation
Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files@@ Coverage Diff @@
## master #1477 +/- ##
============================================
+ Coverage 70.03% 70.08% +0.05%
- Complexity 1656 1658 +2
============================================
Files 32 32
Lines 4268 4272 +4
============================================
+ Hits 2989 2994 +5
+ Misses 1279 1278 -1 ☔ View full report in Codecov by Sentry. |
|
Updated & tested the new docs for Skosmos with |
|
@osma, I updated the documentation during my tests, and managed to get Skosmos working with just However, when doing a final test with
but the STW only displayed the vocabulary general information and failed to show any terms. I re-downloaded my data, but couldn't find what is wrong. Do you know if there's any issue with the STW vocabulary and the latest version of Skosmos? If none, I will troubleshoot it later (by enabling exceptions in the browser console and inspecting the console logs).
Cheers |
|
Thanks a lot, this looks very promising. I think we need to make it clearer in the documentation which files were copied from Jena and how to update them later. I also have some concerns about requiring the use tdbloader, see my comment above. I will test this locally and see how it works, including testing STW that you had some problems with. |
👍
Noted, will try
If you'd like you can wait until I modify it to Thank you Osma! |
Sounds good, I will wait a bit! |
|
Hi @osma, I forgot that the Apache Jena image starts a
In the Docker startup command we define the name of the dataset to be used and location. That's why Apache Jena documentation suggested to create a local dataset with i.e. we cannot start the container and then So I think it is by design, and this would be a bit hard to modify, since the Docker image runs a Maven package that does not include the admin functions. WDYT? |
|
I think one possibility would be to run another |
Hmm, that's a pity. But I understand that sometimes having a minimal Fuseki is desirable. I don't quite understand why using tdbloader2 would be necessary. Normally Fuseki can start with an empty dataset (just the dataset config file) and it will create an empty TDB (and an empty jena-text index, if configured) when it starts up. For example we often wipe the Finto datasets from servers using this kind of procedure:
Could we do the same with the Docker container? Just give it a dataset config file, without actually loading any data. Then after it has started up (and Fuseki has created the dataset directory), we can load data with s-put or plain curl. |
Let me play a bit with Fuseki in the container. I only manage datasets with the UI or via the Jena tools, but if this works it might be actually useful for me to know for other projects, and would definitely simplify our Docker setup for Skosmos users. |
|
Modified the docker-compose command-line to pass the skosmos-web | [Mon Jul 03 19:07:46.267430 2023] [core:notice] [pid 7] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'
skosmos-fuseki | org.apache.jena.sparql.ARQException: Found two matches: var ?root -> http://base/#text_dataset, file:///fuseki/skosmos.ttl#tdb_dataset_readwrite
skosmos-fuseki | at org.apache.jena.sparql.util.QueryExecUtils.getAtMostOne(QueryExecUtils.java:395)
skosmos-fuseki | at org.apache.jena.sparql.util.graph.GraphUtils.findRootByType(GraphUtils.java:213)
skosmos-fuseki | at org.apache.jena.sparql.core.assembler.AssemblerUtils.build(AssemblerUtils.java:138)
skosmos-fuseki | at org.apache.jena.sparql.core.assembler.AssemblerUtils.build(AssemblerUtils.java:132)
skosmos-fuseki | at arq.cmdline.ModAssembler.create(ModAssembler.java:68)
skosmos-fuseki | at arq.cmdline.ModDatasetAssembler.createDataset(ModDatasetAssembler.java:41)
skosmos-fuseki | at org.apache.jena.fuseki.main.cmds.FusekiMain.processModulesAndArgs(FusekiMain.java:370)
skosmos-fuseki | at org.apache.jena.cmd.CmdArgModule.process(CmdArgModule.java:39)
skosmos-fuseki | at org.apache.jena.cmd.CmdMain.mainMethod(CmdMain.java:86)
skosmos-fuseki | at org.apache.jena.cmd.CmdMain.mainRun(CmdMain.java:56)
skosmos-fuseki | at org.apache.jena.cmd.CmdMain.mainRun(CmdMain.java:43)
skosmos-fuseki | at org.apache.jena.fuseki.main.cmds.FusekiMain.run(FusekiMain.java:134)
skosmos-fuseki | at org.apache.jena.fuseki.main.cmds.FusekiMainCmd.main(FusekiMainCmd.java:47)
skosmos-fuseki exited with code 2
Trying a few different configurations before I resort to debugging it in Jena 🙂 EDIT: apache/jena#1950 |
|
Kudos, SonarCloud Quality Gate passed!
|
dockerfiles/config/skosmos.ttl
Outdated
There was a problem hiding this comment.
The diff doesn't look too good on GitHub. Better open the file from this branch without the diff and compare with the Apache Jena docs https://jena.apache.org/documentation/query/text-query.html#text-dataset-assembler
I left the items as close as possible, without really modifying what we had before.
osma
added
the
maintenance
|
Hi @kinow , thanks a lot for this, I finally found time to test it (I was on vacation when you sent this & then had to work on non-Skosmos stuff). I tested building the Docker image. But for some reason it fails to build with an error message detailed logI did run I also tried (EDIT:) I'm running docker commands as non-root, but my user account is in the I have Ubuntu 22.04 and installed docker.io and docker-compose using apt (not snap) from the Ubuntu repos. These are my versions: |
| @@ -0,0 +1 @@ | |||
| Commit: 26153afe9a229bb7a609d5406d899eb240ab385e | |||
There was a problem hiding this comment.
This is more than a little bit cryptic. Maybe this README could explain that these files were copied from the Jena repo?
There was a problem hiding this comment.
+1!
I thought I had written about it somewhere. I will have a look and add it to this README.
There was a problem hiding this comment.
This still needs to be done, right?
|
Hi @osma! 👋 I just tried the following and it worked: $ cd Skosmos
$ git reset --hard kinow/fuseki-docker
$ export JENA_4_VERSION=4.8.0
docker build -t jena-fuseki:$JENA_4_VERSION \
--build-arg JENA_VERSION=$JENA_4_VERSION \
--no-cache dockerfiles/jena-fuseki2-docker
[+] Building 76.9s (26/26) FINISHED docker:default
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 3.57kB 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.17.1 2.2s
=> [internal] load metadata for docker.io/library/eclipse-temurin:17-alp 2.0s
=> [auth] library/eclipse-temurin:pull token for registry-1.docker.io 0.0s
=> [auth] library/alpine:pull token for registry-1.docker.io 0.0s
=> [base 1/11] FROM docker.io/library/eclipse-temurin:17-alpine@sha256 29.6s
=> => resolve docker.io/library/eclipse-temurin:17-alpine@sha256:039f727 0.0s
=> => sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa 3.40MB / 3.40MB 2.9s
=> => sha256:994e83f716a0db024fb37caf707eae5af27172a0fff 9.28MB / 9.28MB 8.0s
=> => sha256:00dcebe8dfc82ba6d7ef1c3aba301f87bfcf69 144.11MB / 144.11MB 26.1s
=> => sha256:039f727ed86402f37524b5d01a129947e2f061d5856901d 320B / 320B 0.0s
=> => sha256:3f30352a87719a81c03c9fad62073e8ec8c44fc736d 1.37kB / 1.37kB 0.0s
=> => sha256:e1e1ae8cfac28d1b75eb2dfa65fdc27070c2e364ee0 4.63kB / 4.63kB 0.0s
=> => extracting sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850 0.1s
=> => sha256:920d29c34aa30c6d8a6a682852c76d9ea24a83c8bab5c45 176B / 176B 3.2s
=> => sha256:6024980cb8b498b9a2ef4e378a9cfe55c4dbd7a0f6c623e 733B / 733B 3.5s
=> => extracting sha256:994e83f716a0db024fb37caf707eae5af27172a0fffb691c 0.5s
=> => extracting sha256:00dcebe8dfc82ba6d7ef1c3aba301f87bfcf69a05c6e9fcc 3.3s
=> => extracting sha256:920d29c34aa30c6d8a6a682852c76d9ea24a83c8bab5c459 0.0s
=> => extracting sha256:6024980cb8b498b9a2ef4e378a9cfe55c4dbd7a0f6c623e7 0.0s
=> [stage-1 1/7] FROM docker.io/library/alpine:3.17.1@sha256:f271e74b17c 5.6s
=> => resolve docker.io/library/alpine:3.17.1@sha256:f271e74b17ced29b915 0.0s
=> => sha256:f271e74b17ced29b915d351685fd4644785c6d1559d 1.64kB / 1.64kB 0.0s
=> => sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87 528B / 528B 0.0s
=> => sha256:042a816809aac8d0f7d7cacac7965782ee2ecac3f21 1.47kB / 1.47kB 0.0s
=> => sha256:8921db27df2831fa6eaa85321205a2470c669b855f3 3.37MB / 3.37MB 5.3s
=> => extracting sha256:8921db27df2831fa6eaa85321205a2470c669b855f3ec95d 0.1s
=> [internal] load build context 0.0s
=> => transferring context: 6.12kB 0.0s
=> [base 2/11] RUN [ "4.8.0" != "" ] || { echo -e '\n**** Set JENA_VERS 1.2s
=> [base 3/11] RUN echo && echo "==== Docker build for Apache Jena Fuse 0.5s
=> [base 4/11] RUN apk add --no-cache curl binutils 1.9s
=> [base 5/11] WORKDIR /fuseki 0.1s
=> [base 6/11] COPY download.sh . 0.0s
=> [base 7/11] RUN chmod a+x download.sh 0.3s
=> [base 8/11] RUN ./download.sh --chksum sha1 "https://repo1.maven.or 12.1s
=> [base 9/11] RUN JDEPS="$(jdeps --multi-release base --print-modul 26.3s
=> [base 10/11] ADD entrypoint.sh . 0.0s
=> [base 11/11] ADD log4j2.properties . 0.0s
=> [stage-1 2/7] COPY --from=base /opt/java-minimal /opt/java-minimal 0.3s
=> [stage-1 3/7] COPY --from=base /fuseki /fuseki 0.2s
=> [stage-1 4/7] WORKDIR /fuseki 0.0s
=> [stage-1 5/7] RUN addgroup -g "1000" "fuseki" && adduser "fuseki" 0.3s
=> [stage-1 6/7] RUN mkdir --parents "/fuseki" && chown -R fuseki /f 0.5s
=> [stage-1 7/7] RUN mkdir -p /fuseki/logs && mkdir -p /fuseki/d 0.3s
=> exporting to image 0.6s
=> => exporting layers 0.6s
=> => writing image sha256:9250a08d4769af1db7cdad8b34ab62516375dedf9309d 0.0s
=> => naming to docker.io/library/jena-fuseki:4.8.0
I'm also running Ubuntu, but jammy, Ubuntu 22.04.3 LTS. And my user is also in the docker group. It might be due to the version of Docker? kinow@ranma:~/Development/php/workspace/Skosmos$ docker --version
Docker version 24.0.5, build ced0996
kinow@ranma:~/Development/php/workspace/Skosmos$ docker compose version
Docker Compose version v2.20.2I had a similar issue at $work this week. In that case, there was a feature in Docker... Compose (I think?) that was only available in the newest version. I am not sure if this is a hard requirement, so I can try to test it with your versions if you think it's better that we support older versions. WDYT? |
|
Thanks @kinow ! First, I also have Ubuntu 22.04 (jammy), not 23.04, sorry for the typo. I edited the comment above to clarify. I removed the old Docker installation that came from the Ubuntu repos and switched to the official docker repo instead. Now I have the exact same versions of I will give a few suggestions for the documentation in this PR, then let's try to get this merged soon-ish to the For the Skosmos 3 PHPUnit and Cypress tests I think we should use the Docker Compose setup for running Skosmos and Fusek, so this will require some changes to the current PHPUnit setup including |
|
Thanks @kinow ! Regarding the documentation:
I tested the instructions once again, and everything seemed to work, so I would be ready to merge this as soon as the Please don't rebase this PR branch anymore as I might be basing some of my work on this! |
Noted!
Thank you!
Ah, sorry, you mentioned that in a review and I forgot to update that. Will add some text in some hours. Thanks again @osma ! |
|
Kudos, SonarCloud Quality Gate passed!
|
|
Done, @osma !
|
|
Excellent, thanks a lot @kinow ! |
Reasons for creating this PR
To use the latest version of Jena Fuseki, and an updated base image.
Link to relevant issue(s), if any
Description of the changes in this PR
This PR uses Apache Jena's official Docker image in our Docker Compose. The Dockerfile is copied, and the commit reference in the README until we find a better way to sync our version used and their releases (e.g. work together to get the image uploaded to some public repository).
curl(no UI with this image)Known problems or uncertainties in this PR
Checklist
.sr-onlyclass, color contrast)