Add Worldcat fetcher

[GlennOlsson]

Add EntryBased fetcher for the Worldcat databse (#2581, #1065)

Problem right now is the API key. Tried requesting a sandbox key, but the request was denied. However I believe you being a trusted organisation might have better luck. You can request a key at this link. The key can be added at this line

Have tested the fetcher and importer with XML content from their test api, which has yield successful results

• Change in CHANGELOG.md described (if applicable)a
• Tests created for changes (if applicable)
• Manually tested changed features in running JabRef (always required)
• Checked documentation: Is the information available and up to date? If not: Issue created at https://github.com/JabRef/user-documentation/issues.

[Siedlerchr]

It would be really great if you could check if WorldCat JSON is possible

[tobiasdiez]

@koppor Could you please try to get an API key for us? That would be perfect. Thanks!

[koppor]

@tobiasdiez I crafted a long answer yesterday, but it was lost. Short answer: Register --> login with university id --> I have to choose "Library of City Stuttgart" (or University of Stuttgart Library) (have to get an account for one of them, but that is doable) --> WorldCat then uses that Library --> I don't want all JabRef traffic going through the Stuttgart library --> Think, we need to add a key configuration and direct all users to https://www.oclc.org/developer/develop/authentication/how-to-request-a-wskey.en.html.

I would be OK for me to add a key for our JUnit tests.

[Siedlerchr]

+1 for integration into the preferences

[tobiasdiez]

@koppor Thanks for investigating this! This is a bit unfortunate as it decreases the user experience (and complexifies the code). Could you maybe contact them (I guess WCAPIRegister@oclc.org)? Maybe they provide access for us that is not bound to a specific university/library.

[koppor]

Reply: JabRef can pay for a subscription. Alternatively, JabRef could ask a library to route traffic through them -- I would need to investigate on the costs for us. - In the current state, I would opt for a preference to enter the WorldCat key.

[Siedlerchr]

@GlennOlsson It would be nice if you could finish this PR.
Here are the things todo:

1. Add a field in the preferences to enter and store the API Key
2. Pass the preferences to the fetcher
3. Switch to the json API. I think it should be easier.

[tobiasdiez]

Concerning the API key, I'm actually in contact with the TU Delft library. But that's a slow progress at the moment for the obvious reasons...

[calixtus]

Remaining issue: The api seems to have changed. Requires a little bit more work.

[koppor]

This PR currently implements WorldCat 1.0 API. This will be sunset at some time: https://www.oclc.org/developer/develop/web-services/worldcat-search-api.en.html. Currently, the version 2.0 is in test, and only "Libraries that maintain both WorldCat Discovery and OCLC Cataloging subscriptions." may use it.

[koppor]

PDF on the WorldCat fetcher: https://matienzo.org/storage/2009/2009-Jun-WorldCatAPI.pdf

Example test:

Manual test using the WEb page: https://www.worldcat.org/search?q=ti%3AMarkdown+ti%3Aarchitectural+ti%3Adecision+ti%3Arecords%3A+ti%3AFormat+ti%3Aand+ti%3Atool+ti%3Asupport&qt=results_page

Similar call (only "Markdown Architectural") to the API:

https://www.worldcat.org/webservices/catalog/search/opensearch?wskey=...ourkey...&q=srw.ti%3DMarkdown%20and%20srw.ti%3DArchitectural

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:diag="http://www.loc.gov/zing/srw/diagnostic/" xmlns:oclcterms="http://purl.org/oclc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
    <title>OCLC Worldcat Search: srw.ti=Markdown and srw.ti=Architectural</title>
    <id>http://worldcat.org/webservices/catalog/search/worldcat/opensearch?q=srw.ti%3DMarkdown+and+srw.ti%3DArchitectural&amp;start=1&amp;count=10&amp;format=atom&amp;wskey=...ourkey...</id>
    <updated>2021-07-07T18:17:09-04:00</updated>
    <subtitle>Search results for srw.ti=Markdown and srw.ti=Architectural at http://worldcat.org/webservices/catalog</subtitle>
    <opensearch:totalResults>1</opensearch:totalResults>
    <opensearch:startIndex>1</opensearch:startIndex>
    <opensearch:itemsPerPage>10</opensearch:itemsPerPage>
    <opensearch:Query role="request" searchTerms="srw.ti=Markdown and srw.ti=Architectural" startPage="1"/>
    <link rel="alternate" href="http://worldcat.org/webservices/catalog/search/worldcat/opensearch?q=srw.ti%3DMarkdown+and+srw.ti%3DArchitectural&amp;start=1&amp;count=10&amp;wskey=yuxeAzDLufk3pxfKFY2SJqUQg1yHGTwX7QkG2zpStsnTlSdhZISwQzdj3rQZ0Czl15y6ZuSMRRl4W2xV" type="text/html"/>
    <link rel="self" href="http://worldcat.org/webservices/catalog/search/worldcat/opensearch?q=srw.ti%3DMarkdown+and+srw.ti%3DArchitectural&amp;start=1&amp;count=10&amp;format=atom&amp;wskey=...ourkey..." type="application/atom+xml"/>
    <link rel="first" href="http://worldcat.org/webservices/catalog/search/worldcat/opensearch?q=srw.ti%3DMarkdown+and+srw.ti%3DArchitectural&amp;start=1&amp;count=10&amp;format=atom&amp;wskey=...ourkey..." type="application/atom+xml"/>
    <link rel="last" href="http://worldcat.org/webservices/catalog/search/worldcat/opensearch?q=srw.ti%3DMarkdown+and+srw.ti%3DArchitectural&amp;start=1&amp;count=10&amp;format=atom&amp;wskey=...ourkey..." type="application/atom+xml"/>
    <link rel="search" type="application/opensearchdescription+xml" href="http://worldcat.org/webservices/catalog/opensearch.description.xml"/>
    <entry>
        <author>
            <name>Abbott, Martin L.</name>
        </author>
        <title>The art of scalability : scalable web architecture, processes, and organizations for the modern enterprise</title>
        <link href="http://worldcat.org/oclc/904905227"/>
        <id>http://worldcat.org/oclc/904905227</id>
        <updated>2019-10-02T23:41:13Z</updated>
        <summary>In The Art of Scalability, Second Edition, leading scalability consultants Martin L. Abbott and Michael T. Fisher cover everything you need to know to smoothly scale products and services for any requirement. This extensively revised edition reflects new technologies, strategies, and lessons, as well as new case studies from the authors' pioneering consulting practice, AKF Partners. Writing for technical and nontechnical decision-makers, Abbott and Fisher cover everything that impacts scalability, including architecture, process, people, organization, and technology. Their insights and recommendations reflect more than thirty years of experience at companies ranging from eBay to Visa, and Salesforce.com to Apple. You'll find updated strategies for structuring organizations to maximize agility and scalability, as well as new insights into the cloud (IaaS/PaaS) transition, NoSQL, DevOps, business metrics, and more. Using this guide's tools and advice, you can systematically clear away obstacles to scalability-and achieve unprecedented IT and business performance.</summary>
        <dc:identifier>urn:ISBN:9780134032801</dc:identifier>
        <dc:identifier>urn:ISBN:0134032802</dc:identifier>
        <dc:identifier>urn:LCCN:2015009317</dc:identifier>
        <oclcterms:recordIdentifier>904905227</oclcterms:recordIdentifier>
    </entry>
</feed>

This is not at all a relation to the queried paper.

[koppor]

The API key customization is done at #7720.

[koppor]

This PR is currently too much work to get it finished.

Main points:

1. Uses old WorldCat API 1.0. Not sure how future proof this is.
2. Querying using the 1.0 API lead to wrong results (see Add Worldcat fetcher #6035 (comment))
3. The PR misses the important feature to search for a query string (see Support Worldcat.org as fetcher #1065 (comment)). The current implemented functionality is to add BibTeX data to an existing entry.

Thus, I close this PR and hope that some other external contributor will take over.

Am sorry that I did not guide straight from the beginning through JabRef's different fetcher possibilities (the documentation at https://devdocs.jabref.org/advanced-reading/fetchers is far from being complete) and stressing that the most demanded feature is that a user searches for new entries.