Skip to content

Commit

Permalink
Introduces test http server
Browse files Browse the repository at this point in the history
- Resources
  - /libraries
  - /libraries/{id} (both (embedded) BibTeX as string and CSL data)
- Documentation / code style
  - Refined BibEntry class (JavaDoc, builder)
  - Comments to InternalField
  - move StyleTester to "test" module - package ...testutils/interactive...
- Makes use of Jersey, Grizzly
- Makes use of HK2 as dependency injection framework
- Introduces "application/x-bibtex-library-csl+json" mimetype
- Preparation for client/server sync (BibEntryDTO)
- Minor
  - Made class "JabRefItemDataProvider" more visible
  - Encoding of a .bib file can now be asked for externally
  - Resorts modle-info.java
  - Fixes typo in NetworkTabViewModel
- Installs SLF4J logging router: If a tool uses java commons logging, tinylog now also handles these logs
  • Loading branch information
koppor committed Apr 8, 2023
1 parent 85bacd9 commit 6580c14
Show file tree
Hide file tree
Showing 42 changed files with 1,257 additions and 250 deletions.
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ gradlew text eol=lf

# .bib files have to be written using OS specific line endings to enable our tests working
*.bib text !eol
# Exception: The files used for the http server test - they should have linux line endings
src/test/resources/org/jabref/http/server/*.bib text eol=lf

# Citavi needs to be LF line ending
# This overwrites the setting of "*.bib"
Expand Down
20 changes: 20 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,8 @@ dependencies {
implementation "org.tinylog:tinylog-api:2.6.1"
implementation "org.tinylog:slf4j-tinylog:2.6.1"
implementation "org.tinylog:tinylog-impl:2.6.1"
// route all requests to java.util.logging to SLF4J (which in turn routes to tinylog)
implementation 'org.slf4j:jul-to-slf4j:2.0.7'

implementation 'de.undercouch:citeproc-java:3.0.0-beta.2'

Expand All @@ -199,6 +201,24 @@ dependencies {

implementation group: 'net.harawata', name: 'appdirs', version: '1.2.1'

// JAX-RS implemented by Jersey
// API
implementation 'jakarta.ws.rs:jakarta.ws.rs-api:3.1.0'
// Implementation of the API
implementation 'org.glassfish.jersey.core:jersey-server:3.1.1'
// injection framework
implementation 'org.glassfish.jersey.inject:jersey-hk2:3.1.1'
implementation 'org.glassfish.hk2:hk2-api:2.6.1'
// testImplementation 'org.glassfish.hk2:hk2-testing:3.0.4'
// implementation 'org.glassfish.hk2:hk2-testing-jersey:3.0.4'
// testImplementation 'org.glassfish.hk2:hk2-junitrunner:3.0.4'
// HTTP server
// implementation 'org.glassfish.jersey.containers:jersey-container-netty-http:3.1.1'
implementation 'org.glassfish.jersey.containers:jersey-container-grizzly2-http:3.1.1'
testImplementation 'org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-grizzly2:3.1.1'
// Allow objects "magically" to be mapped to JSON using GSON
// implementation 'org.glassfish.jersey.media:jersey-media-json-gson:3.1.1'

testImplementation 'io.github.classgraph:classgraph:4.8.157'
testImplementation 'org.junit.jupiter:junit-jupiter:5.9.2'
testImplementation 'org.junit.platform:junit-platform-launcher:1.9.2'
Expand Down
53 changes: 53 additions & 0 deletions docs/decisions/0027-http-return-bibtex-string.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
nav_order: 27
parent: Decision Records
---
<!-- we need to disable MD025, because we use the different heading "ADR Template" in the homepage (see above) than it is foreseen in the template -->
<!-- markdownlint-disable-next-line MD025 -->
# Return BibTeX string and CSL Item JSON in the API

## Context and Problem Statement

In the context of an http server, when a http client `GETs` a JSON data structure containing BibTeX data, which format should that have?

## Considered Options

* Offer both, BibTeX string and CSL JSON
* Return BibTeX as is as string
* Convert BibTeX to JSON

## Decision Outcome

Chosen option: "Offer both, BibTeX string and CSL JSON", because there are many browser libraries out there being able to parse BibTeX. Thus, we don't need to convert it.

## Pros and Cons of the Options

### Offer both, BibTeX string and CSL JSON

- Good, because this follows "Backend for Frontend"
- Good, because Word Addin works seamless with the data provided (and does not need another dependency)
- Good, because other clients can work with BibTeX data
- Bad, because two serializations have to be kept

### Return BibTeX as is as string

- Good, because we don't need to think about any conversion
- Bad, because it is unclear how to ship BibTeX data where the entry is dependent on
- Bad, because client needs add additional parsing logic

### Convert BibTeX to JSON

More thought has to be done when converting to JSON.
There seems to be a JSON format from [@citation-js/plugin-bibtex](https://www.npmjs.com/package/@citation-js/plugin-bibtex).
We could do an additional self-made JSON format, but this increases the number of available JSON serializations for BibTeX.

- Good, because it could flatten BibTeX data (example: `author = first # " and " # second`)
- Bad, because conversion is difficult in BibTeX special cases. For instance, if Strings are used (example: `author = first # " and " # second`) and one doesn't want to flatten ("normalize") this.

## More Information

Existing JavaScript BibTeX libraries:

* [bibtex-js](https://github.com/digitalheir/bibtex-js)
* [bibtexParseJS](https://github.com/ORCID/bibtexParseJs)
* [@citation-js/plugin-bibtex](https://www.npmjs.com/package/@citation-js/plugin-bibtex)
72 changes: 44 additions & 28 deletions src/main/java/module-info.java
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
requires afterburner.fx;
requires com.jfoenix;
requires de.saxsys.mvvmfx;
requires reactfx;
requires de.saxsys.mvvmfx.validation;
requires org.fxmisc.flowless;

requires org.kordamp.ikonli.core;
requires org.kordamp.ikonli.javafx;
Expand All @@ -38,6 +41,7 @@

// Logging
requires org.slf4j;
requires jul.to.slf4j;
requires org.tinylog.api;
requires org.tinylog.api.slf4j;
requires org.tinylog.impl;
Expand All @@ -46,47 +50,58 @@
with org.jabref.gui.logging.GuiWriter,
org.jabref.gui.logging.ApplicationInsightsWriter;

// Preferences and XML
requires java.prefs;

// Annotations (@PostConstruct)
requires jakarta.annotation;
requires jakarta.inject;

// http server and client exchange
requires java.net.http;
requires jakarta.ws.rs;
requires grizzly.framework;

// data mapping
requires jakarta.xml.bind;
requires jdk.xml.dom;
requires com.google.gson;
requires com.fasterxml.jackson.databind;
requires com.fasterxml.jackson.dataformat.yaml;
requires com.fasterxml.jackson.datatype.jsr310;
// needs to be loaded here as it's otherwise not found at runtime
requires org.glassfish.jaxb.runtime;
requires jdk.xml.dom;

// Annotations (@PostConstruct)
requires jakarta.annotation;
// dependency injection using HK2
requires org.glassfish.hk2.api;

// Microsoft application insights
requires applicationinsights.core;
requires applicationinsights.logging.log4j2;

// Libre Office
requires org.libreoffice.uno;

// Other modules
requires com.google.common;
requires jakarta.inject;
requires reactfx;
requires commons.cli;
requires com.github.tomtung.latex2unicode;
requires fastparse;
requires jbibtex;
requires citeproc.java;
requires de.saxsys.mvvmfx.validation;
requires com.google.gson;
// http clients
requires unirest.java;
requires org.apache.httpcomponents.httpclient;
requires org.jsoup;
requires org.apache.commons.csv;
requires io.github.javadiffutils;
requires java.string.similarity;

// SQL databases
requires ojdbc10;
requires org.postgresql.jdbc;
requires org.mariadb.jdbc;
uses org.mariadb.jdbc.credential.CredentialPlugin;

// Apache Commons and other (similar) helper libraries
requires commons.cli;
requires org.apache.commons.csv;
requires org.apache.commons.lang3;
requires org.antlr.antlr4.runtime;
requires org.fxmisc.flowless;
requires com.google.common;
requires io.github.javadiffutils;
requires java.string.similarity;

requires com.github.tomtung.latex2unicode;
requires fastparse;

requires jbibtex;
requires citeproc.java;

requires pdfbox;
requires xmpbox;
Expand All @@ -95,26 +110,27 @@
requires flexmark;
requires flexmark.util.ast;
requires flexmark.util.data;
requires com.h2database.mvstore;

// fulltext search
requires org.apache.lucene.core;
// In case the version is updated, please also adapt SearchFieldConstants#VERSION to the newly used version
uses org.apache.lucene.codecs.lucene94.Lucene94Codec;

requires org.apache.lucene.queryparser;
uses org.apache.lucene.queryparser.classic.MultiFieldQueryParser;
requires org.apache.lucene.analysis.common;
requires org.apache.lucene.highlighter;

requires com.fasterxml.jackson.databind;
requires com.fasterxml.jackson.dataformat.yaml;
requires com.fasterxml.jackson.datatype.jsr310;
requires net.harawata.appdirs;
requires com.sun.jna;
requires com.sun.jna.platform;

requires org.eclipse.jgit;
uses org.eclipse.jgit.transport.SshSessionFactory;
uses org.eclipse.jgit.lib.GpgSigner;

// other libraries
requires com.h2database.mvstore;
requires org.antlr.antlr4.runtime;
requires org.libreoffice.uno;

}
21 changes: 14 additions & 7 deletions src/main/java/org/jabref/cli/Launcher.java
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import org.apache.commons.cli.ParseException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.bridge.SLF4JBridgeHandler;
import org.tinylog.configuration.Configuration;

/**
Expand All @@ -47,6 +48,7 @@ public class Launcher {
private static String[] ARGUMENTS;

public static void main(String[] args) {
routeLoggingToSlf4J();
ARGUMENTS = args;
addLogToDisk();
try {
Expand Down Expand Up @@ -85,6 +87,11 @@ public static void main(String[] args) {
}
}

private static void routeLoggingToSlf4J() {
SLF4JBridgeHandler.removeHandlersForRootLogger();
SLF4JBridgeHandler.install();
}

/**
* This needs to be called as early as possible. After the first log write, it
* is not possible to alter
Expand All @@ -93,10 +100,10 @@ public static void main(String[] args) {
private static void addLogToDisk() {
Path directory = Path.of(AppDirsFactory.getInstance()
.getUserDataDir(
OS.APP_DIR_APP_NAME,
"logs",
OS.APP_DIR_APP_AUTHOR))
.resolve(new BuildInfo().version.toString());
OS.APP_DIR_APP_NAME,
"logs",
OS.APP_DIR_APP_AUTHOR))
.resolve(new BuildInfo().version.toString());
try {
Files.createDirectories(directory);
} catch (IOException e) {
Expand Down Expand Up @@ -187,9 +194,9 @@ private static void clearOldSearchIndices() {
&& !path.equals(currentIndexPath)) {
LOGGER.info("Deleting out-of-date fulltext search index at {}.", path);
Files.walk(path)
.sorted(Comparator.reverseOrder())
.map(Path::toFile)
.forEach(File::delete);
.sorted(Comparator.reverseOrder())
.map(Path::toFile)
.forEach(File::delete);
}
}
} catch (IOException e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ public void addCertificateFile() {
.build();

dialogService.showFileOpenDialog(fileDialogConfiguration).ifPresent(certPath -> SSLCertificate.fromPath(certPath).ifPresent(sslCertificate -> {
if (!trustStoreManager.isCertificateExist(formatCustomAlias(sslCertificate.getSHA256Thumbprint()))) {
if (!trustStoreManager.certificateExists(formatCustomAlias(sslCertificate.getSHA256Thumbprint()))) {
customCertificateListProperty.add(CustomCertificateViewModel.fromSSLCertificate(sslCertificate)
.setPath(certPath.toAbsolutePath().toString()));
} else {
Expand Down
6 changes: 6 additions & 0 deletions src/main/java/org/jabref/http/MediaType.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package org.jabref.http;

public class MediaType {
public static final String BIBTEX = "application/x-bibtex";
public static final String JSON_CSL_ITEM = "application/x-bibtex-library-csl+json";
}
71 changes: 71 additions & 0 deletions src/main/java/org/jabref/http/dto/BibEntryDTO.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
package org.jabref.http.dto;

import java.io.IOException;
import java.io.StringWriter;

import org.jabref.logic.bibtex.BibEntryWriter;
import org.jabref.logic.bibtex.FieldWriter;
import org.jabref.logic.bibtex.FieldWriterPreferences;
import org.jabref.logic.exporter.BibWriter;
import org.jabref.model.database.BibDatabaseMode;
import org.jabref.model.entry.BibEntry;
import org.jabref.model.entry.BibEntryTypesManager;
import org.jabref.model.entry.SharedBibEntryData;

import com.google.common.base.MoreObjects;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* The data transfer object (DTO) for an BibEntry
*
* @param sharingMetadata the data used for sharing
* @param userComments the comments before the BibTeX entry
* @param citationKey the citation key (duplicated from BibEntry to ease processing by the client)
* @param bibtex the BibEntry as BibTeX string (see ADR-0027 for more information, why we don't use a HashMap / JSON)
*/
public record BibEntryDTO(SharedBibEntryData sharingMetadata, String userComments, String citationKey, String bibtex) implements Comparable<BibEntryDTO> {

public static final Logger LOGGER = LoggerFactory.getLogger(BibEntryDTO.class);

public BibEntryDTO(BibEntry bibEntry, BibDatabaseMode bibDatabaseMode, FieldWriterPreferences fieldWriterPreferences, BibEntryTypesManager bibEntryTypesManager) {
this(bibEntry.getSharedBibEntryData(),
bibEntry.getUserComments(),
bibEntry.getCitationKey().orElse(""),
convertToString(bibEntry, bibDatabaseMode, fieldWriterPreferences, bibEntryTypesManager)
);
}

private static String convertToString(BibEntry entry, BibDatabaseMode bibDatabaseMode, FieldWriterPreferences fieldWriterPreferences, BibEntryTypesManager bibEntryTypesManager) {
StringWriter rawEntry = new StringWriter();
BibWriter bibWriter = new BibWriter(rawEntry, "\n");
BibEntryWriter bibtexEntryWriter = new BibEntryWriter(new FieldWriter(fieldWriterPreferences), bibEntryTypesManager);
try {
bibtexEntryWriter.write(entry, bibWriter, bibDatabaseMode);
} catch (IOException e) {
LOGGER.warn("Problem creating BibTeX entry.", e);
return "error";
}
return rawEntry.toString();
}

@Override
public int compareTo(BibEntryDTO o) {
int sharingComparison = sharingMetadata.compareTo(o.sharingMetadata);
if (sharingComparison != 0) {
return sharingComparison;
}
LOGGER.error("Comparing equal DTOs");
return 0;
}

@Override
public String toString() {
return MoreObjects.toStringHelper(this)
.add("sharingMetadata", sharingMetadata)
.add("userComments", userComments)
.add("citationkey", citationKey)
.add("bibtex", bibtex)
.toString();
}
}
18 changes: 18 additions & 0 deletions src/main/java/org/jabref/http/dto/GsonFactory.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package org.jabref.http.dto;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.glassfish.hk2.api.Factory;

public class GsonFactory implements Factory<Gson> {
@Override
public Gson provide() {
return new GsonBuilder()
.setPrettyPrinting()
.create();
}

@Override
public void dispose(Gson instance) {
}
}
Loading

0 comments on commit 6580c14

Please sign in to comment.