Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend ArXiv fetcher results by using data from related DOIs #9170

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
bcda94d
Added new Fetcher that mimics ArXiv fetcher, but also gets new fields…
thiagocferr Sep 22, 2022
dfe4ed6
Added feature flag between new and old behavior in preferences menu (…
thiagocferr Sep 23, 2022
f31b2b0
Merged both new and old ArXiv fecthers into one
thiagocferr Sep 23, 2022
d661a70
Removed feature flag / button, implemented prioritized fields from DO…
thiagocferr Sep 27, 2022
9820b9a
Get even more info from user-assigned DOIs
thiagocferr Sep 28, 2022
690b8fe
Made modifications to ArXivFetcher and its testing, passing all of them
thiagocferr Sep 29, 2022
cbfa44d
Undone several String changes from "ArXivFetcher" back to "ArXiv"
thiagocferr Sep 29, 2022
a221027
Some fixes to tests broken by changes from the new ArXiv fetcher
thiagocferr Sep 30, 2022
b0fdfb1
Parallelize process of field infusion for search queries
thiagocferr Oct 2, 2022
566a96d
Parallelized all extended ArXiv fetching process to reduce on process…
thiagocferr Oct 3, 2022
cac0b95
Added API Rate Limiting for async calls to DOI API (DOI Content Negoc…
thiagocferr Oct 16, 2022
3fe4dbf
Merge branch 'main' into 9092-arXiv-fetcher-import-more-information-u…
thiagocferr Oct 17, 2022
fabf5ac
Replaced commas from ArXiv keywords (Category Taxonomy) by slashes
thiagocferr Oct 17, 2022
2dfeefa
Small changes to logging messages, exception handling, comments, etc.
thiagocferr Oct 19, 2022
a562d99
Added some tests, comments and docuemntation
thiagocferr Oct 20, 2022
430af28
QUICKFIX: avoinding potential NullPointerException on response header
thiagocferr Oct 20, 2022
b97b28a
QUICKFIX: change in selected fields afected CompositeIdFetcher tests
thiagocferr Oct 20, 2022
2d931ed
QUICKFIX: forgot to include entry on CHANGELOG
thiagocferr Oct 20, 2022
71764b3
Merge branch 'main' into 9092-arXiv-fetcher-import-more-information-u…
Siedlerchr Nov 1, 2022
a2dacdd
Merge branch 'JabRef:main' into 9092-arXiv-fetcher-import-more-inform…
thiagocferr Nov 2, 2022
0c7d307
Merge branch 'JabRef:main' into 9092-arXiv-fetcher-import-more-inform…
thiagocferr Nov 3, 2022
704086f
Modified keyword duplication removal by calling existing method inste…
thiagocferr Nov 3, 2022
e5fa5f9
Update WebFetchersTest.java to include error on Logger
thiagocferr Nov 7, 2022
c27a3ad
Small coding style changes
thiagocferr Nov 8, 2022
786b3a5
Merge branch '9092-arXiv-fetcher-import-more-information-using-DOI' o…
thiagocferr Nov 8, 2022
f24620e
Forgot a comma while commiting suggested modifications from Github UI
thiagocferr Nov 8, 2022
9ca2f3c
Another set of small corrections
thiagocferr Nov 9, 2022
3e66c66
QUICKFIX: fixed wrong output for CompositeIdFetcherTest on ArXiv calls
thiagocferr Nov 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 0 additions & 14 deletions .idea/runConfigurations/JabRef_Main.xml

This file was deleted.

1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ Note that this project **does not** adhere to [Semantic Versioning](http://semve
- We fixed an issue where a message about changed metadata would occur on saving although nothing changed. [#9159](https://github.com/JabRef/jabref/issues/9159)
- When adding or editing a subgroup it is placed w.r.t. to alphabetical ordering rather than at the end. [koppor#577](https://github.com/koppor/jabref/issues/577)
- We modified the Directory of Open Access Books (DOAB) fetcher so that it will now also fetch the ISBN when possible. [#8708](https://github.com/JabRef/jabref/issues/8708)
- ArXiv fetcher now retrieves additional data from related DOIs (both ArXiv and user-assigned). [#9170](https://github.com/JabRef/jabref/pull/9170)
- Groups context menu now shows appropriate options depending on number of subgroups. [koppor#579](https://github.com/koppor/jabref/issues/579)
- We changed the color of found text from red to high-contrast colors (background: yellow; font color: purple). [koppor#552](https://github.com/koppor/jabref/issues/552)
- We modified the "Delete file" dialog and add the full file path to the dialog text. The file path in the title was changed to file name only. [koppor#534](https://github.com/koppor/jabref/issues/534)
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/org/jabref/gui/externalfiles/ImportHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
import org.jabref.logic.importer.ImportFormatReader;
import org.jabref.logic.importer.ImportFormatReader.UnknownFormatImport;
import org.jabref.logic.importer.ParseException;
import org.jabref.logic.importer.fetcher.ArXiv;
import org.jabref.logic.importer.fetcher.ArXivFetcher;
import org.jabref.logic.importer.fetcher.DoiFetcher;
import org.jabref.logic.importer.fileformat.BibtexParser;
import org.jabref.logic.l10n.Localization;
Expand Down Expand Up @@ -301,7 +301,7 @@ private List<BibEntry> fetchByDOI(DOI doi) throws FetcherException {

private List<BibEntry> fetchByArXiv(ArXivIdentifier arXivIdentifier) throws FetcherException {
LOGGER.info("Found arxiv identifier in clipboard");
Optional<BibEntry> entry = new ArXiv(preferencesService.getImportFormatPreferences()).performSearchById(arXivIdentifier.getNormalizedWithoutVersion());
Optional<BibEntry> entry = new ArXivFetcher(preferencesService.getImportFormatPreferences()).performSearchById(arXivIdentifier.getNormalizedWithoutVersion());
return OptionalUtil.toList(entry);
}
}
2 changes: 1 addition & 1 deletion src/main/java/org/jabref/gui/maintable/CellFactory.java
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ public CellFactory(PreferencesService preferencesService, UndoManager undoManage
TABLE_ICONS.put(new UnknownField("citeseerurl"), icon);

icon = IconTheme.JabRefIcons.WWW;
// icon.setToolTipText(Localization.lang("Open") + " ArXiv URL");
// icon.setToolTipText(Localization.lang("Open") + " ArXivFetcher URL");
TABLE_ICONS.put(StandardField.EPRINT, icon);

icon = IconTheme.JabRefIcons.DOI;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import java.util.Optional;

import org.jabref.logic.importer.fetcher.ArXiv;
import org.jabref.logic.importer.fetcher.ArXivFetcher;
import org.jabref.logic.importer.fetcher.DoiFetcher;
import org.jabref.logic.importer.fetcher.isbntobibtex.DoiToBibtexConverterComIsbnFetcher;
import org.jabref.logic.importer.fetcher.isbntobibtex.EbookDeIsbnFetcher;
Expand All @@ -27,7 +27,7 @@ public Optional<BibEntry> performSearchById(String identifier) throws FetcherExc
}
Optional<ArXivIdentifier> arXivIdentifier = ArXivIdentifier.parse(identifier);
if (arXivIdentifier.isPresent()) {
return new ArXiv(importFormatPreferences).performSearchById(arXivIdentifier.get().getNormalized());
return new ArXivFetcher(importFormatPreferences).performSearchById(arXivIdentifier.get().getNormalized());
}
Optional<ISBN> isbn = ISBN.parse(identifier);
if (isbn.isPresent()) {
Expand Down
12 changes: 6 additions & 6 deletions src/main/java/org/jabref/logic/importer/WebFetchers.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import org.jabref.logic.importer.fetcher.ACMPortalFetcher;
import org.jabref.logic.importer.fetcher.ACS;
import org.jabref.logic.importer.fetcher.ApsFetcher;
import org.jabref.logic.importer.fetcher.ArXiv;
import org.jabref.logic.importer.fetcher.ArXivFetcher;
import org.jabref.logic.importer.fetcher.AstrophysicsDataSystem;
import org.jabref.logic.importer.fetcher.BiodiversityLibrary;
import org.jabref.logic.importer.fetcher.CiteSeer;
Expand Down Expand Up @@ -70,7 +70,7 @@ public static Optional<IdBasedFetcher> getIdBasedFetcherForField(Field field, Im
.addRetryFetcher(new EbookDeIsbnFetcher(importFormatPreferences))
.addRetryFetcher(new DoiToBibtexConverterComIsbnFetcher(importFormatPreferences));
} else if (field == EPRINT) {
fetcher = new ArXiv(importFormatPreferences);
fetcher = new ArXivFetcher(importFormatPreferences);
} else {
return Optional.empty();
}
Expand Down Expand Up @@ -98,7 +98,7 @@ public static Optional<IdFetcher<? extends Identifier>> getIdFetcherForField(Fie
*/
public static SortedSet<SearchBasedFetcher> getSearchBasedFetchers(ImportFormatPreferences importFormatPreferences, ImporterPreferences importerPreferences) {
SortedSet<SearchBasedFetcher> set = new TreeSet<>(Comparator.comparing(WebFetcher::getName));
set.add(new ArXiv(importFormatPreferences));
set.add(new ArXivFetcher(importFormatPreferences));
set.add(new INSPIREFetcher(importFormatPreferences));
set.add(new GvkFetcher());
set.add(new MedlineFetcher());
Expand Down Expand Up @@ -129,7 +129,7 @@ public static SortedSet<SearchBasedFetcher> getSearchBasedFetchers(ImportFormatP
public static SortedSet<IdBasedFetcher> getIdBasedFetchers(ImportFormatPreferences importFormatPreferences,
ImporterPreferences importerPreferences) {
SortedSet<IdBasedFetcher> set = new TreeSet<>(Comparator.comparing(WebFetcher::getName));
set.add(new ArXiv(importFormatPreferences));
set.add(new ArXivFetcher(importFormatPreferences));
set.add(new AstrophysicsDataSystem(importFormatPreferences, importerPreferences));
set.add(new IsbnFetcher(importFormatPreferences)
.addRetryFetcher(new EbookDeIsbnFetcher(importFormatPreferences))
Expand Down Expand Up @@ -177,7 +177,7 @@ public static SortedSet<EntryBasedFetcher> getEntryBasedFetchers(ImporterPrefere
public static SortedSet<IdFetcher<? extends Identifier>> getIdFetchers(ImportFormatPreferences importFormatPreferences) {
SortedSet<IdFetcher<?>> set = new TreeSet<>(Comparator.comparing(WebFetcher::getName));
set.add(new CrossRef());
set.add(new ArXiv(importFormatPreferences));
set.add(new ArXivFetcher(importFormatPreferences));
return set;
}

Expand All @@ -194,7 +194,7 @@ public static Set<FulltextFetcher> getFullTextFetchers(ImportFormatPreferences i
fetchers.add(new ScienceDirect(importerPreferences));
fetchers.add(new SpringerLink(importerPreferences));
fetchers.add(new ACS());
fetchers.add(new ArXiv(importFormatPreferences));
fetchers.add(new ArXivFetcher(importFormatPreferences));
fetchers.add(new IEEE(importFormatPreferences, importerPreferences));
fetchers.add(new ApsFetcher());

Expand Down
Loading