Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EndNote XML Exporter + Rehaul Importer #11157

Merged
merged 98 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
1d3a3ac
Fix LICENSE link
subhramit Apr 5, 2024
316753f
Add Endnote XML Exporter
subhramit Apr 7, 2024
eed6822
Merge branch 'main' of https://github.com/JabRef/jabref into fix-for-…
subhramit Apr 7, 2024
5da187a
Fix Exporter and Importer - stage 1
subhramit Apr 7, 2024
1b211e8
Review changes-1
subhramit Apr 7, 2024
c84619e
Changelog addition
subhramit Apr 7, 2024
06a314e
Changelog line location change
subhramit Apr 7, 2024
cad0157
Merge branch 'main' of https://github.com/JabRef/jabref into fix-for-…
subhramit Apr 8, 2024
a595cdd
Merge branch 'JabRef:main' into fix-for-issue-11137
subhramit Apr 8, 2024
571e226
Review changes-2 (Majorly formatted XML)
subhramit Apr 8, 2024
ef7a69b
Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…
subhramit Apr 8, 2024
7108400
Merge branch 'main' into fix-for-issue-11137
subhramit Apr 8, 2024
354d514
Add test
subhramit Apr 8, 2024
d0f2a37
Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…
subhramit Apr 8, 2024
100ebf4
Fix single entry export test
subhramit Apr 8, 2024
07b2616
Fix ref-type in test
subhramit Apr 8, 2024
720d0ac
Fix ref-type in test
subhramit Apr 8, 2024
a0999c4
Ordered export item types
subhramit Apr 9, 2024
ba01435
Adapt test to Journal Entry type
subhramit Apr 9, 2024
1d26c65
Shift factory initialization to constructor, remove code duplicacy & …
subhramit Apr 9, 2024
a091fb7
Add fields - pagetotal, month, day, etc.
subhramit Apr 9, 2024
b252d8c
Add series/secondary title
subhramit Apr 9, 2024
3309cd9
Update test
subhramit Apr 9, 2024
f05675a
Update test - add tertiary title, fix ref no
subhramit Apr 9, 2024
2b4cab9
Review changes -3 (LinkedHashMap)
subhramit Apr 9, 2024
f56dadb
Fix LinkedHashMap implementation
subhramit Apr 9, 2024
3490c45
Merge branch 'main' into fix-for-issue-11137
subhramit Apr 9, 2024
04c9bc8
Attempt fix of LHM
subhramit Apr 9, 2024
3542169
Major rehaul: Conform to Endnote DTD, add missing fields, fix missing…
subhramit Apr 13, 2024
3b43e2a
Merge branch 'main' of https://github.com/JabRef/jabref into fix-for-…
subhramit Apr 13, 2024
4b00fd7
Fix exporter test
subhramit Apr 13, 2024
71c18e3
Preserve order of entry type mappings
subhramit Apr 13, 2024
d9361f7
minor refactor
subhramit Apr 14, 2024
05f797e
rewriteRun
subhramit Apr 14, 2024
3dead24
Fix ordering of entry types
subhramit Apr 14, 2024
899a12d
Test run success, TBD
subhramit Apr 14, 2024
73a4984
Fix exporter test
subhramit Apr 14, 2024
9df4fdb
Fix exporter test-2
subhramit Apr 14, 2024
fdf1357
Fix exporter test-3
subhramit Apr 14, 2024
1b29089
Fix exporter test-4
subhramit Apr 14, 2024
b9bc72f
Fix order of fields in exporter, adapt test(1)
subhramit Apr 14, 2024
0cfb995
Extract test resources
Siedlerchr Apr 14, 2024
ad037e3
fix merge conflicts
Siedlerchr Apr 14, 2024
1ad2189
Exporter Beta, Importer TODO!
subhramit Apr 14, 2024
83202b8
Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…
subhramit Apr 14, 2024
125e8a9
Exporter and Importer - final
subhramit Apr 15, 2024
58f5529
Merge branch 'main' into fix-for-issue-11137
subhramit Apr 16, 2024
4ffe0e7
Parse style children content (Fixes Importer tests)
subhramit Apr 16, 2024
39603a7
Remove grouped temporary test files
subhramit Apr 16, 2024
45a9388
Merge branch 'main' of https://github.com/JabRef/jabref into fix-for-…
subhramit Apr 16, 2024
1d17607
Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…
subhramit Apr 16, 2024
6f152b1
Removed debug printlns
subhramit Apr 16, 2024
d26978d
Fix exporter tests
subhramit Apr 16, 2024
5dbd152
Rename test file
subhramit Apr 16, 2024
b73dee1
Merge branch 'main' into fix-for-issue-11137
subhramit Apr 16, 2024
d7b9cc6
Fix parsing of 'related-urls'
subhramit Apr 16, 2024
e2cf097
Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…
subhramit Apr 16, 2024
aa6c6d3
Merge branch 'main' into fix-for-issue-11137
subhramit Apr 16, 2024
5447845
Remove citation keys from exporter test bib
subhramit Apr 16, 2024
a76b51e
Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…
subhramit Apr 16, 2024
f8b671d
Fix newlines in 'notes' field of importer test files
subhramit Apr 16, 2024
9b36179
Fix Title and Number overwrite
subhramit Apr 16, 2024
aa7edcf
Fix overwrite of title and number without losing info
subhramit Apr 16, 2024
8e1e7dd
Fix handling of multi-part titles & unclean titles
subhramit Apr 16, 2024
51961f4
Ad PDF suffix
subhramit Apr 16, 2024
69f7f30
Fix Importer tests
subhramit Apr 17, 2024
1eff5b9
Fix Importer tests-2
subhramit Apr 17, 2024
0b8fec4
Fix Exporter and Importer tests - trailing :PDF
subhramit Apr 17, 2024
bd87bc3
Fix Round-trip test: preceeding ':' in pdf files
subhramit Apr 17, 2024
cf37b6e
Fix typo in changelog
subhramit Apr 17, 2024
f85ce6c
Merge branch 'main' into fix-for-issue-11137
subhramit Apr 17, 2024
1732b72
More relevant changelog entry
subhramit Apr 17, 2024
c528071
Merge branch 'fix-for-issue-11137' of https://github.com/subhramit/ja…
subhramit Apr 17, 2024
74ea7e4
Use HashMap Java data type
koppor Apr 17, 2024
a78ffc4
Extract methods
koppor Apr 17, 2024
f596f3d
Extract method
koppor Apr 17, 2024
e0c18af
Restructure test
koppor Apr 17, 2024
375614b
Use "native" method for keyword list parsing
koppor Apr 17, 2024
ab70573
Fix usage of entry.getAuthorTitleYear
koppor Apr 17, 2024
61372df
Add hint to getCitationKey() usage
koppor Apr 17, 2024
a4df60b
Add some helper thing
koppor Apr 17, 2024
9b66b5e
Use author parsing
koppor Apr 17, 2024
2fe3d62
Improve (and use JOURNAL only)
koppor Apr 17, 2024
4532e28
Fix obsolete .toString()
koppor Apr 17, 2024
e0964a8
Removed commented alt-title import
subhramit Apr 17, 2024
8a081ed
Added back 'aalto-xml' comments and IS_COALESCING property
subhramit Apr 17, 2024
a3bb94d
Adapt Importer test bib's for alt-title
subhramit Apr 17, 2024
7d9763d
Add export for alt-title
subhramit Apr 17, 2024
fdb465d
Removed TODO comment (complete)
subhramit Apr 17, 2024
96b34bc
Merge branch 'main' into fix-for-issue-11137
subhramit Apr 17, 2024
eba7afd
Cleanup operation if journal matches alt-title
subhramit Apr 17, 2024
60f3ae2
Update expected bib wrt cleanup
subhramit Apr 17, 2024
07bf22c
Update expected xml titles to include secondary-titles
subhramit Apr 17, 2024
ca8e5f7
Handle case of Books/Non-article journal -> booktitle
subhramit Apr 17, 2024
08393ba
Journal (alt-periodical) check for book
subhramit Apr 17, 2024
6af33a3
Add cleanup for booktitle==alt-title
subhramit Apr 17, 2024
0527ec7
More efficient cleanup
subhramit Apr 17, 2024
794376f
Review changes - minor comments
subhramit Apr 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv
### Added

- We added a new keyboard shortcut <kbd>ctrl</kbd> + <kbd>,</kbd> to open the preferences. [#11154](https://github.com/JabRef/jabref/pull/11154)
- We added an exporter for Endnote XML format. [#11137](https://github.com/JabRef/jabref/issues/11137)

### Changed

Expand Down
209 changes: 209 additions & 0 deletions src/main/java/org/jabref/logic/exporter/EndnoteXmlExporter.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
package org.jabref.logic.exporter;

import java.io.BufferedWriter;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.stream.Collectors;

import javax.xml.stream.XMLOutputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamWriter;

import org.jabref.logic.importer.AuthorListParser;
import org.jabref.logic.util.StandardFileType;
import org.jabref.model.database.BibDatabaseContext;
import org.jabref.model.entry.Author;
import org.jabref.model.entry.BibEntry;
import org.jabref.model.entry.field.StandardField;
import org.jabref.model.entry.types.EntryType;
import org.jabref.model.entry.types.IEEETranEntryType;
import org.jabref.model.entry.types.StandardEntryType;

public class EndnoteXmlExporter extends Exporter {

private static final String ENDNOTE_XML_VERSION = "20.1";
private static final Map<EntryType, String> EXPORT_ITEM_TYPE = Map.ofEntries(
Map.entry(StandardEntryType.Article, "Journal Article"),
Map.entry(StandardEntryType.Book, "Book"),
Map.entry(StandardEntryType.InBook, "Book Section"),
Map.entry(StandardEntryType.InCollection, "Book Section"),
Map.entry(StandardEntryType.InProceedings, "Conference Paper"),
Map.entry(StandardEntryType.Conference, "Conference"),
Map.entry(StandardEntryType.MastersThesis, "Thesis"),
Map.entry(StandardEntryType.PhdThesis, "Thesis"),
Map.entry(StandardEntryType.Proceedings, "Conference Proceedings"),
Map.entry(StandardEntryType.TechReport, "Report"),
Map.entry(StandardEntryType.Unpublished, "Manuscript"),
Map.entry(IEEETranEntryType.Patent, "Patent"),
Map.entry(StandardEntryType.Online, "Web Page"),
Map.entry(IEEETranEntryType.Electronic, "Electronic Article"),
Map.entry(StandardEntryType.Misc, "Generic")
);

private static final Map<EntryType, String> EXPORT_REF_NUMBER = EXPORT_ITEM_TYPE.entrySet().stream()
.collect(Collectors.toMap(Map.Entry::getKey, entry -> Integer.toString(EXPORT_ITEM_TYPE.entrySet().stream()
.map(Map.Entry::getValue)
.collect(Collectors.toList())
.indexOf(entry.getValue()) + 1)));

private final XMLOutputFactory xmlOutputFactory;
private final AuthorListParser authorListParser;

public EndnoteXmlExporter() {
super("endnote", "EndNote XML", StandardFileType.XML);
this.xmlOutputFactory = XMLOutputFactory.newFactory();
this.authorListParser = new AuthorListParser();
}

@Override
public void export(BibDatabaseContext databaseContext, Path file, List<BibEntry> entries) throws Exception {
Objects.requireNonNull(databaseContext);
Objects.requireNonNull(file);
Objects.requireNonNull(entries);

if (entries.isEmpty()) {
return;
}

try (BufferedWriter writer = Files.newBufferedWriter(file, StandardCharsets.UTF_8)) {
XMLStreamWriter xml = xmlOutputFactory.createXMLStreamWriter(writer);
xml.writeStartDocument(StandardCharsets.UTF_8.name(), "1.0");
HoussemNasri marked this conversation as resolved.
Show resolved Hide resolved
xml.writeStartElement("xml");
xml.writeStartElement("records");

for (BibEntry entry : entries) {
writeEntry(entry, xml);
}

xml.writeEndElement(); // records
xml.writeEndElement(); // xml
xml.writeEndDocument();
xml.close();
} catch (IOException | XMLStreamException e) {
throw new SaveException(e);
}
}

private void writeEntry(BibEntry entry, XMLStreamWriter xml) throws XMLStreamException {
xml.writeStartElement("record");

// Write the necessary fields and elements
writeField(xml, "database", "endnote.enl", Map.of("name", "My EndNote Library.enl", "path", "/path/to/My EndNote Library.enl"));
writeField(xml, "source-app", "JabRef", Map.of("name", "JabRef", "version", ENDNOTE_XML_VERSION));
writeField(xml, "rec-number", entry.getId(), null);

xml.writeStartElement("foreign-keys");
xml.writeStartElement("key");
xml.writeAttribute("app", "EN");
xml.writeCharacters(entry.getId());
xml.writeEndElement(); // key
xml.writeEndElement(); // foreign-keys

writeField(xml, "ref-type", EXPORT_REF_NUMBER.getOrDefault(entry.getType(), "Generic"), Map.of("name", EXPORT_ITEM_TYPE.getOrDefault(entry.getType(), "Generic")));

writeContributors(entry, xml);
writeField(xml, "titles", null, Map.of(), entry.getField(StandardField.TITLE).orElse(""), "title");
writeField(xml, "periodical", null, Map.of(), entry.getField(StandardField.JOURNAL).orElse(""), "full-title");
writeField(xml, "tertiary-title", entry.getField(StandardField.BOOKTITLE).orElse(""), null);
writeField(xml, "pages", entry.getField(StandardField.PAGES).orElse(""), null);
writeField(xml, "volume", entry.getField(StandardField.VOLUME).orElse(""), null);
writeField(xml, "number", entry.getField(StandardField.NUMBER).orElse(""), null);
writeField(xml, "dates", null, Map.of(), entry.getField(StandardField.YEAR).orElse(""), "year");
writePublisher(entry, xml);
writeField(xml, "isbn", entry.getField(StandardField.ISBN).orElse(""), null);
writeField(xml, "abstract", entry.getField(StandardField.ABSTRACT).orElse(""), null);
writeField(xml, "notes", entry.getField(StandardField.NOTE).orElse(""), null);
writeField(xml, "urls", null, Map.of(), entry.getField(StandardField.URL).orElse(""), "web-urls");
writeField(xml, "electronic-resource-num", entry.getField(StandardField.DOI).orElse(""), null);

xml.writeEndElement(); // record
}

private void writeField(XMLStreamWriter xml, String name, String value, Map<String, String> attributes) throws XMLStreamException {
if (value != null && !value.isEmpty()) {
xml.writeStartElement(name);
if (attributes != null) {
for (Map.Entry<String, String> attribute : attributes.entrySet()) {
xml.writeAttribute(attribute.getKey(), attribute.getValue());
}
}
xml.writeStartElement("style");
xml.writeAttribute("face", "normal");
xml.writeAttribute("font", "default");
xml.writeAttribute("size", "100%");
xml.writeCharacters(value);
xml.writeEndElement(); // style
xml.writeEndElement(); // name
}
}

private void writeField(XMLStreamWriter xml, String name, String value, Map<String, String> attributes, String childValue, String childElementName) throws XMLStreamException {
xml.writeStartElement(name);
if (attributes != null) {
for (Map.Entry<String, String> attribute : attributes.entrySet()) {
xml.writeAttribute(attribute.getKey(), attribute.getValue());
}
}
if (childValue != null && !childValue.isEmpty()) {
xml.writeStartElement(childElementName);
xml.writeStartElement("style");
xml.writeAttribute("face", "normal");
xml.writeAttribute("font", "default");
xml.writeAttribute("size", "100%");
xml.writeCharacters(childValue);
xml.writeEndElement(); // style
xml.writeEndElement(); // childElementName
}
xml.writeEndElement(); // name
}

private void writeContributors(BibEntry entry, XMLStreamWriter xml) throws XMLStreamException {
entry.getField(StandardField.AUTHOR).ifPresent(authors -> {
try {
xml.writeStartElement("contributors");
xml.writeStartElement("authors");
for (Author author : authorListParser.parse(authors).getAuthors()) {
HoussemNasri marked this conversation as resolved.
Show resolved Hide resolved
xml.writeStartElement("author");
xml.writeStartElement("style");
xml.writeAttribute("face", "normal");
xml.writeAttribute("font", "default");
xml.writeAttribute("size", "100%");
xml.writeCharacters(author.getFamilyGiven(false));
xml.writeEndElement(); // style
xml.writeEndElement(); // author
}
xml.writeEndElement(); // authors
xml.writeEndElement(); // contributors
} catch (XMLStreamException e) {
throw new RuntimeException("Error writing contributors", e);
}
});
}

private void writePublisher(BibEntry entry, XMLStreamWriter xml) throws XMLStreamException {
String publisher = entry.getField(StandardField.PUBLISHER).orElse("");
String address = entry.getField(StandardField.ADDRESS).orElse("");
if (!publisher.isEmpty() || !address.isEmpty()) {
xml.writeStartElement("publisher");
if (!publisher.isEmpty()) {
writeField(xml, "publisher", publisher, null);
}
if (!address.isEmpty()) {
xml.writeStartElement("place");
xml.writeStartElement("style");
xml.writeAttribute("face", "normal");
xml.writeAttribute("font", "default");
xml.writeAttribute("size", "100%");
xml.writeCharacters(address);
xml.writeEndElement(); // style
xml.writeEndElement(); // place
}
xml.writeEndElement(); // publisher
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ public static ExporterFactory create(PreferencesService preferencesService,
exporters.add(new XmpPdfExporter(xmpPreferences));
exporters.add(new EmbeddedBibFilePdfExporter(bibDatabaseMode, entryTypesManager, fieldPreferences));
exporters.add(new CffExporter());
exporters.add(new EndnoteXmlExporter());

// Now add custom export formats
exporters.addAll(customFormats);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,9 @@ private void parseRecord(XMLStreamReader reader, List<BibEntry> bibItems, String
case "number" -> {
parseStyleContent(reader, fields, StandardField.NUMBER, elementName);
}
case "tertiary-title" -> {
parseStyleContent(reader, fields, StandardField.BOOKTITLE, elementName);
}
case "dates" -> {
parseYear(reader, fields);
}
Expand Down Expand Up @@ -207,7 +210,9 @@ private static EntryType convertRefNameToType(String refName) {
case "book section" -> StandardEntryType.InBook;
case "book" -> StandardEntryType.Book;
case "report" -> StandardEntryType.Report;
// case "journal article" -> StandardEntryType.Article;
case "conference paper" -> StandardEntryType.InProceedings;
case "conference" -> StandardEntryType.Conference;
case "conference proceedings" -> StandardEntryType.Proceedings;
default -> StandardEntryType.Article;
};
}
Expand Down
Loading