Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CFF import/export and craft a round-trip test #10995

Merged
merged 44 commits into from
Mar 21, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
f437d58
issue #10993 - feat: added ability to parse preferred-citation field …
jeanprbt Mar 8, 2024
b5298df
issue #10993 - feat: added all fields of JabRef/CITATION.cff to CffIm…
jeanprbt Mar 8, 2024
a6b62e1
issue #10993 - feat: rewrote CffExporter to parse Software, Dataset t…
jeanprbt Mar 11, 2024
cd94d33
issue #10993 - feat: added keywords and unknown fields support
jeanprbt Mar 11, 2024
ca0f887
issue #10993 - feat: added round-trip test
jeanprbt Mar 11, 2024
0b1b578
issue #10993 - doc: updated CHANGELOG.md
jeanprbt Mar 11, 2024
d32d26f
Merge branch 'JabRef:main' into issue/10993
jeanprbt Mar 15, 2024
56bf7e7
Convert RemoveBracesFormatterTest to @ParameterizedTest (#11033)
koppor Mar 15, 2024
c4b2328
Importing of BibDesk Groups and Linked Files (#10968)
Frequinzy Mar 17, 2024
57f8a63
Speed up failure reporting (#11030)
koppor Mar 17, 2024
7a4be6d
Fixes Zotero file handling for absolute paths (#11038)
Siedlerchr Mar 18, 2024
7abf13d
Change copy-paste function to handle string constants (follow up PR) …
Siedlerchr Mar 18, 2024
9587520
Bump gittools/actions from 0.13.4 to 1.1.1 (#11039)
dependabot[bot] Mar 18, 2024
1ec6a6e
Bump com.googlecode.plist:dd-plist from 1.23 to 1.28 (#11040)
dependabot[bot] Mar 18, 2024
930a9b4
Bump org.apache.pdfbox:xmpbox from 3.0.1 to 3.0.2 (#11041)
dependabot[bot] Mar 18, 2024
5858598
Bump com.dlsc.gemsfx:gemsfx from 2.2.0 to 2.4.0 (#11044)
dependabot[bot] Mar 18, 2024
7cb8885
Bump org.apache.pdfbox:fontbox from 3.0.1 to 3.0.2 (#11042)
dependabot[bot] Mar 18, 2024
342cb24
Keep enclosing braces of authors (#11034)
koppor Mar 18, 2024
5ab2a81
Improve citation relations (#11016)
ror3d Mar 18, 2024
7a269d4
issue #10993 - doc: updated CHANGELOG.md
jeanprbt Mar 11, 2024
8a8434a
Merge branch 'main' into issue/10993
jeanprbt Mar 18, 2024
008472b
fix: fixed unit tests not passing due to name changes in Author inter…
jeanprbt Mar 18, 2024
6f925ec
feat: changed CFFExporter to use YAML library snakeyaml instead (#10995)
jeanprbt Mar 18, 2024
5a60aff
feat: added support for references and ALL possible CFF fields in imp…
jeanprbt Mar 18, 2024
8fbdf26
Merge branch 'main' into issue/10993
jeanprbt Mar 18, 2024
5e697a2
fix: added requested changes (#10995)
jeanprbt Mar 19, 2024
88c42b8
fix: task rewriteDryRun fixed to pass by removing test in BibEntryTest
jeanprbt Mar 19, 2024
e1b1665
Merge branch 'main' into issue/10993
jeanprbt Mar 19, 2024
9271368
refactor: deleted useless methods in CffImporter (#10995)
jeanprbt Mar 19, 2024
69245be
doc: added decision MADR document for cff export (#10995)
jeanprbt Mar 19, 2024
ad2d600
feat: add a cites or related relationship between imported entries in…
jeanprbt Mar 20, 2024
6978078
Merge branch 'main' into issue/10993
jeanprbt Mar 20, 2024
ca9c0dc
doc: updated MADR decision document for cff export to pass markdownli…
jeanprbt Mar 20, 2024
359237d
fix: fixed round-trip test to use mock citatioKeyPatternPreferences c…
jeanprbt Mar 20, 2024
a8518b7
fix: fixed MADR document for CFF export decision to pass Jekyll CI ch…
jeanprbt Mar 20, 2024
0264c03
fix: fixed requested changes (#10995)
jeanprbt Mar 20, 2024
c4bc13c
feat: finished CFFExporter logic and crafted working round-trip test …
jeanprbt Mar 21, 2024
de27eef
Merge branch 'main' into issue/10993
jeanprbt Mar 21, 2024
2450c80
fix: fixed typos in MADR decision doc for CFF export and refactore Im…
jeanprbt Mar 21, 2024
8d72c5f
Some code beautification
koppor Mar 21, 2024
bf9ff8b
Use existing method getEntryLinkList
koppor Mar 21, 2024
447632b
Use getEntryLinkList
koppor Mar 21, 2024
c43d14a
Use JabRef's Date class for parsing
koppor Mar 21, 2024
60904da
Fix indentation in new line
calixtus Mar 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv
- We enhanced the dialog for adding new fields in the content selector with a selection box containing a list of standard fields. [#10912](https://github.com/JabRef/jabref/pull/10912)
- We store the citation relations in an LRU cache to avoid bloating the memory and out-of-memory exceptions. [#10958](https://github.com/JabRef/jabref/issues/10958)
- Keywords filed are now displayed as tags. [#10910](https://github.com/JabRef/jabref/pull/10910)
- We use another exporter for CFF format and the importer parses more fields. [#10993](https://github.com/JabRef/jabref/issues/10993)
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved

### Fixed

Expand Down
242 changes: 242 additions & 0 deletions src/main/java/org/jabref/logic/exporter/CffExporter.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
package org.jabref.logic.exporter;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.time.LocalDate;
import java.time.Year;
import java.time.YearMonth;
import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeParseException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;

import org.jabref.logic.util.OS;
import org.jabref.logic.util.StandardFileType;
import org.jabref.model.database.BibDatabaseContext;
import org.jabref.model.entry.Author;
import org.jabref.model.entry.AuthorList;
import org.jabref.model.entry.BibEntry;
import org.jabref.model.entry.field.BiblatexSoftwareField;
import org.jabref.model.entry.field.Field;
import org.jabref.model.entry.field.StandardField;
import org.jabref.model.entry.field.UnknownField;
import org.jabref.model.entry.types.EntryType;
import org.jabref.model.entry.types.StandardEntryType;

/**
* Exporter for exporting in CFF format.
*/
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved
class CffExporter extends Exporter {
public CffExporter() {
super("cff", "CFF", StandardFileType.CFF);
}

@Override
public void export(BibDatabaseContext databaseContext, Path file, List<BibEntry> entries) throws Exception {
koppor marked this conversation as resolved.
Show resolved Hide resolved
Objects.requireNonNull(databaseContext);
Objects.requireNonNull(file);
Objects.requireNonNull(entries);

if (entries.isEmpty()) { // Do not export if no entries to export -- avoids exports with only template text
return;
}

try (AtomicFileWriter ps = new AtomicFileWriter(file, StandardCharsets.UTF_8)) {
ps.write("# YAML 1.2" + OS.NEWLINE);
ps.write("---" + OS.NEWLINE);
ps.write("cff-version: 1.2.0" + OS.NEWLINE);

for (BibEntry entry : entries) {
// Retrieve all fields
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved
Map<Field, String> entryMap = entry.getFieldMap();
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved

// Compulsory message field
String message = entryMap.getOrDefault(StandardField.COMMENT,
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved
"If you use this software, please cite it using the metadata from this file.");
ps.write("message: " + message + OS.NEWLINE);
entryMap.remove(StandardField.COMMENT);

// Compulsory title field
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved
String title = entryMap.getOrDefault(StandardField.TITLE, "No title specified.");
ps.write("title: " + "\"" + title + "\"" + OS.NEWLINE);
entryMap.remove(StandardField.TITLE);

// Compulsory authors field
List<Author> authors = AuthorList.parse(entryMap.getOrDefault(StandardField.AUTHOR, ""))
.getAuthors();
writeAuthors(ps, authors, false);
entryMap.remove(StandardField.AUTHOR);

// Type
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved
Map<EntryType, String> typeMap = getTypeMappings();
EntryType entryType = entry.getType();
boolean pref = false;
switch (entryType) {
case StandardEntryType.Software ->
ps.write("type: software" + OS.NEWLINE);
case StandardEntryType.Dataset ->
ps.write("type: dataset" + OS.NEWLINE);
default -> {
if (typeMap.containsKey(entryType)) {
pref = true;
ps.write("preferred-citation:" + OS.NEWLINE);
ps.write(" type: " + typeMap.get(entryType) + OS.NEWLINE);
writeAuthors(ps, authors, true);
ps.write(" title: " + "\"" + title + "\"" + OS.NEWLINE);
}
}
}

// Keywords
String keywords = entryMap.getOrDefault(StandardField.KEYWORDS, null);
if (keywords != null) {
ps.write(pref ? " " : "");
ps.write("keywords:" + OS.NEWLINE);
for (String keyword : keywords.split(",\\s*")) {
ps.write(pref ? " " : "");
ps.write(" - " + keyword + OS.NEWLINE);
}
}
entryMap.remove(StandardField.KEYWORDS);

// Date
String date = entryMap.getOrDefault(StandardField.DATE, null);
if (date != null) {
writeDate(ps, date, pref);
}
entryMap.remove(StandardField.DATE);

// Fields
Map<Field, String> fieldMap = getFieldMappings();
for (Field field : entryMap.keySet()) {
if (fieldMap.containsKey(field)) {
ps.write(pref ? " " : "");
ps.write(fieldMap.get(field) + ": " + "\"" + entryMap.get(field) + "\"" + OS.NEWLINE);
} else if (field instanceof UnknownField) {
ps.write(pref ? " " : "");
ps.write(field.getName() + ": " + "\"" + entryMap.get(field) + "\"" + OS.NEWLINE);
}
}
}
} catch (IOException ex) {
throw new SaveException(ex);
}
}

private void writeAuthors(AtomicFileWriter ps, List<Author> authors, boolean pref) throws Exception {
try {
ps.write(pref ? " " : "");
ps.write("authors:");
if (authors.isEmpty()) {
ps.write(pref ? " " : "");
ps.write(" No author specified.");
} else {
ps.write(OS.NEWLINE);
}
for (Author author : authors) {
boolean hyphen = false;
if (author.getLast().isPresent()) {
ps.write(pref ? " " : "");
ps.write(" - family-names: " + author.getLast().get() + OS.NEWLINE);
hyphen = true;
}
if (author.getFirst().isPresent()) {
ps.write(pref ? " " : "");
ps.write(hyphen ? " " : " - ");
ps.write("given-names: " + author.getFirst().get() + OS.NEWLINE);
hyphen = true;
}
if (author.getVon().isPresent()) {
ps.write(pref ? " " : "");
ps.write(hyphen ? " " : " - ");
ps.write("name-particle: " + author.getVon().get() + OS.NEWLINE);
hyphen = true;
}
if (author.getJr().isPresent()) {
ps.write(pref ? " " : "");
ps.write(hyphen ? " " : " - ");
ps.write("name-suffix: " + author.getJr().get() + OS.NEWLINE);
}
}
} catch (IOException ex) {
throw new SaveException(ex);
}
}

private void writeDate(AtomicFileWriter ps, String dateField, boolean pref) throws Exception {
StringBuilder builder = new StringBuilder();
String formatString = "yyyy-MM-dd";
try {
DateTimeFormatter formatter = DateTimeFormatter.ofPattern(formatString);
LocalDate date = LocalDate.parse(dateField, DateTimeFormatter.ISO_LOCAL_DATE);
builder.append(pref ? " " : "").append("date-released: ").append(date.format(formatter));
} catch (DateTimeParseException e) {
if (pref) {
try {
formatString = "yyyy-MM";
DateTimeFormatter formatter = DateTimeFormatter.ofPattern(formatString);
YearMonth yearMonth = YearMonth.parse(dateField, formatter);
int month = yearMonth.getMonth().getValue();
int year = yearMonth.getYear();
builder.append(" month: ").append(month).append(OS.NEWLINE);
builder.append(" year: ").append(year).append(OS.NEWLINE);
} catch (DateTimeParseException f) {
try {
formatString = "yyyy";
DateTimeFormatter formatter = DateTimeFormatter.ofPattern(formatString);
int year = Year.parse(dateField, formatter).getValue();
builder.append(" year: ").append(year).append(OS.NEWLINE);
} catch (DateTimeParseException g) {
builder.append(" issue-date: ").append(dateField).append(OS.NEWLINE);
}
}
}
}
try {
ps.write(builder.toString());
} catch (IOException ex) {
throw new SaveException(ex);
}
}

private Map<EntryType, String> getTypeMappings() {
Map<EntryType, String> typeMappings = new HashMap<>();
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved
typeMappings.put(StandardEntryType.Article, "article");
typeMappings.put(StandardEntryType.Conference, "article");
typeMappings.put(StandardEntryType.Book, "book");
typeMappings.put(StandardEntryType.Booklet, "pamphlet");
typeMappings.put(StandardEntryType.InProceedings, "conference-paper");
typeMappings.put(StandardEntryType.Proceedings, "proceedings");
typeMappings.put(StandardEntryType.Misc, "misc");
typeMappings.put(StandardEntryType.Manual, "manual");
typeMappings.put(StandardEntryType.Report, "report");
typeMappings.put(StandardEntryType.TechReport, "report");
typeMappings.put(StandardEntryType.Unpublished, "unpublished");
return typeMappings;
}

private Map<Field, String> getFieldMappings() {
Map<Field, String> fieldMappings = new HashMap<>();
jeanprbt marked this conversation as resolved.
Show resolved Hide resolved
fieldMappings.put(StandardField.TITLE, "title");
fieldMappings.put(StandardField.VERSION, "version");
fieldMappings.put(StandardField.DOI, "doi");
fieldMappings.put(BiblatexSoftwareField.LICENSE, "license");
fieldMappings.put(BiblatexSoftwareField.REPOSITORY, "repository");
fieldMappings.put(StandardField.URL, "url");
fieldMappings.put(StandardField.ABSTRACT, "abstract");
fieldMappings.put(StandardField.COMMENT, "message");
fieldMappings.put(StandardField.DATE, "date-released");
fieldMappings.put(StandardField.KEYWORDS, "keywords");
fieldMappings.put(StandardField.MONTH, "month");
fieldMappings.put(StandardField.YEAR, "year");
fieldMappings.put(StandardField.JOURNAL, "journal");
fieldMappings.put(StandardField.ISSUE, "issue");
fieldMappings.put(StandardField.VOLUME, "volume");
fieldMappings.put(StandardField.NUMBER, "number");
return fieldMappings;
}
}

Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,14 @@ public static ExporterFactory create(PreferencesService preferencesService,
exporters.add(new TemplateExporter("MIS Quarterly", "misq", "misq", "misq", StandardFileType.RTF, layoutPreferences, saveOrder));
exporters.add(new TemplateExporter("CSL YAML", "yaml", "yaml", null, StandardFileType.YAML, layoutPreferences, saveOrder, BlankLineBehaviour.DELETE_BLANKS));
exporters.add(new TemplateExporter("Hayagriva YAML", "hayagrivayaml", "hayagrivayaml", null, StandardFileType.YAML, layoutPreferences, saveOrder, BlankLineBehaviour.DELETE_BLANKS));
exporters.add(new TemplateExporter("CFF", "cff", "cff", null, StandardFileType.CFF, layoutPreferences, saveOrder, BlankLineBehaviour.DELETE_BLANKS));
exporters.add(new OpenOfficeDocumentCreator());
exporters.add(new OpenDocumentSpreadsheetCreator());
exporters.add(new MSBibExporter());
exporters.add(new ModsExporter());
exporters.add(new XmpExporter(xmpPreferences));
exporters.add(new XmpPdfExporter(xmpPreferences));
exporters.add(new EmbeddedBibFilePdfExporter(bibDatabaseMode, entryTypesManager, fieldPreferences));
exporters.add(new CffExporter());

// Now add custom export formats
exporters.addAll(customFormats);
Expand Down
Loading
Loading