Wikidata

Wikidata

Fetches and transforms data from Wikidata, using the Wikibase-SDK library to simplify querying Wikidata and the results.

get /wiki/{wikidata}

This route will run if a record has a wikidata field, which is currently limited to People (and company) records only.

{
  "data": {
    "___comment": "### WARNING ### - <enhancement> tags are experimental, please do not aggregate",
    "type": "people",
    "id": "cp20600",
    "attributes": {  
      "wikidata": "https://www.wikidata.org/wiki/Q312"
    },
   
  },
} 

Creating a Wikibase instance

const WBK = require('wikibase-sdk');
const wdk = WBK({
  instance: 'https://www.wikidata.org',
  sparqlEndpoint: 'https://query.wikidata.org/sparql'
});
module.exports = wdk;

Note that the Wikibase-SDK documentation states that this is the old way of creating an instance, but it was the only way that worked.

This instance can use the getEntities method, passing in values to configure the results

Further Wikibase documentation

result = await wbk.getEntities(
            fieldId,
            'en',
            ['labels'],
            'json'
          );

The output of this will be a url that can be consumed with an api call

https://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q312&format=json&redirects=no&languages=en%7Cfr%7Cde&props=info%7Cclaims

config

Most of the useful information is in the claims array, but the labels property, if passed to getEntities function can be useful if you just want to pull out the text of value at the top level, rather than the QCode.

Inside the claims array, each item contains a datavalue field, which is usually the most useful property needed, and can be another QCode for a making a nested call, or can be a datestring or another value. The qualifiers array contains extra contextual information, such as for a job title, it could contain company associated with the job, and the date range in the role.

{
  "entities": {
    "Q312": {
      "pageid": "",
      "ns": "",
      "title": "",
      "lastrevid": "",
      "modified": "",
      "type": "",
      "id": "",
      "claims": {
        "P1297": [
          {
            "mainsnak": {
              "snaktype": "",
              "property": "",
              "hash": "",
              "datavalue": {
                "value": "",
                "type": ""
              },
              "datatype": ""
            },
            "type": "",
            "qualifiers": {
              "P580": [
                {
                  "snaktype": "",
                  "property": "",
                  "hash": "",
                  "datavalue": {
                    "value": {
                    },
                    "type": ""
                  },
                  "datatype": ""
                }
              ]
            }
          }
        ]
      }
    }
  }
}

use cases

• Awards
• VIAF ID
• Employees
• CEOS
• Position Held
• many more

utility functions

Check wikidataQueries for util functions that transform data, handle nested or single values, generate strings, transform qualifier arrays for extra context on fields, date formatting, and more.

The project is config driven, and uses parameters in wikibasePropertiesConfig.js for functionality. This file contains a list of properties we want to pull out if available in a record's Wikidata, and the parameters allow them to be displayed and/or transformed in various ways.

module.exports = {
  'Official Name': { property: 'P1448', action: [match, display] },
  'Employer(s)': {
    property: 'P108',
    action: [nest, match, display, displayLinked, context, list]
  },
  Industry: { property: 'P452', action: [nest, match, list] },
  'Position Held': {
    property: 'P39',
    action: [nest, match, displayLinked, context, list]
  },
  'Date of Birth': {
    property: 'P569',
    action: [match, display, hide]
  },
  'Owned By': {
    property: 'P127',
    action: [nest, display, displayLinked, context]
  },
  
  'Has Subsidiary': {
    property: 'P355',
    action: [nest, display, displayLinked, displayLinked, context]
  },
  'Chief Executive Officers': {
    property: 'P169',
    action: [nest, display, displayLinked, displayLinked, context]
  },
  'Notable Work': { property: 'P800', action: [nest, display] },
  'VIAF ID': { property: 'P214', action: [display, isExternalLink] }
};

caching

The Wikidata code is hooked up to Redis, specifically Catbox-Redis to allow it to be used with Hapi.JS.

Each Wikidata result is cached on first load, and then pulled from the cache on subsequent loads.

There is also an optional query string parameter: /wikidata?clear=true

This is for dropping individual records from the cache if required. It can be run from api testing tools like Postman.