Highlight

the open access detection made by Unpaywall is highly reliable (> 99 %).

Context

This is a pursuit of a work made by Aaron Tay Open Access Indicators and the importance of optimising your Institutional repository for discovery in Unpaywal
In 2020 Open Access indicators at institution level are growing (see for example CWTS Leiden Ranking 2020 ). Most of them are using Unpaywall to know if a publication is open access or not.
The goal here is to represent the limits of unpaywall.

Steps :

1. get bibliographic data from Scopus (or something else)
2. run and adapt 0_enrich_scopus_w_upw.py to get open access status for each publications
3. run and adapt 1_compare_w_hal.py to compare with HAL repository
4. run and adapt 2_draw_graphics.py to represent what cannot be treated with unpaywall

Results

1. Reproducing Aaron works : For a publication mark as closed in unpaywall, can I find an open access version in my institutional repository ?

At first, 16 publications were concerned

Source 2014-2017	% of open access
Scopus + Unpaywall	2855 / 5597 = 51.01%
Scopus + Unpaywall + institutionnal repository (HAL)	2871 / 5597 = 51.29%

However, after a manual verification only 2 publications were really in that case

DOI	Problem
10.1016/j.respe.2017.03.133	embargo forever
10.1016/j.jtemb.2016.11.006	embargo forever
10.1007/s10552-016-0832-4	embargo forever
10.1016/j.spinee.2013.08.017	embargo forever
10.1136/jnnp-2013-306799	embargo forever
10.1016/j.vaccine.2014.11.045	embargo forever
10.1016/j.bulcan.2016.06.005	embargo forever
10.4230/LIPIcs.STACS.2016.52	isn't in Unpaywall
10.1002/pssc.201700181	isn't in Unpaywall
10.4074/S0761898017002072	isn't in Unpaywall
10.1002/2015GL066170	loaded 3 August
10.1002/2016GL070620	loaded 3 August
10.1002/2017GL074353	loaded 3 August
10.1088/2041-8205/807/2/L29	loaded 5 August
10.1016/j.ijepes.2014.07.064	upw missed it
10.1038/nature23266	upw missed it

2. How much pubications cannot be treated with unpaywall ?

Unpaywall can treat publications with a Crossref DOI, cf. Unpaywall FAQ.
From the 5597 publications finded in Scopus 4% cannot be treated with Unpaywall