The sections below list open data provision by the Scottish Government and its directorates, agencies, NDPBS etc.
12 August 2020
Trying to work out the business units within the structure of Scottish Government is a significant challenge in itself. Attempting to then establish which have published open data, and what those data sets are, and how they are licensed, is almost an impossible task. If my checking, and arithmetic are right, then of 147 discrete business units, only 27 have published open data and 120 have published none.
So we can say with some confidence that the issue with findability of data raised in Feb 2019 is unchanged: there being no central portal for open data in the Scottish public sector or even for Scottish Government. Searching the main Scottish Government website for open data yields 633 results, none of which are links to data on the first four screenfuls. I didn't go deeper than that.
It is important to note that the whole of the gov.scot website is covered by an OGL open licence. Sadly this is not true for most of the public sector in Scotland.
A list of open data resources in the Scottish Government.
Data is broken into three types - APIs, open data, and linked open data.
| Key | Definition |
|---|---|
| A | API |
| L | Linked Open Data |
| O | Open Data |
The Scottish Government's Statistics have a very good portal with 295 Data Sets from multiple organisational-providers. This is up by 46 datasets on last year and includes a two new organisations: The Care Inspectorate and Registers of Scotland. The latter, so far, has no datasets on the portal.
I've heard numerous comments from people who find the interface challenging but there are some very helpful guides provided.
| Organisation | URL | Datasets | Change | Type |
|---|---|---|---|---|
| Accountant in Bankruptcy | URL | 3 | - | L |
| Care Inspectorate | URL | 3 | +3 | L |
| National Records of Scotland | URL | 29 | +3 | L |
| NHS Information Services Division | URL | 36 | +8 | L |
| Registers of Scotland | URL | 0 | - | L |
| Revenue Scotland | URL | 3 | +1 | L |
| Scottish Fire and rescue Servcive | URL | 1 | - | L |
| Scottish Government | URL | 191 | +25 | L |
| Scottish Natural Heritage | URL | 2 | - | O |
| SEPA | URL | 1 | - | O |
| Transport Scotland | URL | 8 | - | O |
| Visit Scotland | URL | 6 | - | O |
| All | URL | 295 | +46 | O |
- The Scottish Spatial Data Infrastructure. This is a metadata repository for Inspire data, but does not publish open data.
- Scottish Indices of Multiple Deprivation: map and Data is on the Statistics Portal mentioned above, specifically here
- Spatial Hub - still has 4 open datasets; no change on 2019.
According to this page there are forty-three directorates of the Scottish Government, up from thirty-three at the start of 2019. These are listed below. So far it appears that few produce open data. Wikipedia, which counts 42 directorates, groups these into Directorates-General but this grouping is not obvious from the SG site.
Incidentally, why isn't there Linked Open Data published about the structure of the Scottish Government with unique identifiers and permanent URIs for each? Surely there is a small project for a civil servant to get stuck into this on Wikidata for a day?
Addtitionally, when accessing the 286 Scottish Government data sets (see the URL in the table above), below the Explore bar there are links (depending on the dataset) to various 'concept folders' such as housing many of which must surely align with some of the list below (e.g. Housing and Social Justice Directorate) but unpicking which have published Open Data and which have not appears almost impossible at the moment.
- Advanced Learning and Science Directorate
- Agriculture and Rural Delivery Directorate
- Budget and Public Spending Directorate
- Chief Economist Directorate
- Chief Medical Officer Directorate
- Chief Nursing Officer Directorate
- Children and Families Directorate
- Communications, Ministerial Support and Facilities Directorate
- Community Health and Social Care Directorate
- Constitution and Cabinet Directorate
- Culture, Tourism and Major Events Directorate
- Digital Directorate
- Early Learning and Childcare Programme Directorate
- Economic Development Directorate
- Energy and Climate Change Directorate
- Environment and Forestry Directorate
- External Affairs Directorate
- Fair Work, Employability and Skills Directorate
- Financial Management Directorate
- Financial Strategy Directorate
- Health Finance, Corporate Governance and Value Directorate
- Health Performance and Delivery Directorate
- Health Workforce, Leadership and Service Reform Directorate
- Healthcare Quality and Improvement Directorate
- Housing and Social Justice Directorate
- Internal Audit and Assurance Directorate
- International Trade and Investment Directorate
- Justice Directorate
- Learning Directorate
- Legal Services (Solicitor to the Scottish Government)
- Local Government and Communities Directorate
- Marine Scotland Directorate (MS)
- Mental Health Directorate
- Office of the Chief Executive NHS Scotland
- Parliamentary Counsel Office
- People Directorate
- Performance and Strategic Outcomes Directorate
- Population Health Directorate
- Safer Communities Directorate
- Scottish National Investment Bank Directorate
- Scottish Procurement and Property Directorate
- Social Security Directorate
- Taxation and Fiscal Sustainability Directorate
There are three old datasets about homecare (2010-2012) made available here. It took someone who knows about this to find them and let me know about their existence. This is on the old Scottishs Government Website. The new one is here. The main page is clearly marked at, the foot, "Crown Copyright." However, clicking on that take us to this page. "Re-using Crown copyright material. You may use and re-use the information featured on this website (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence."
The figures below were checked in March 2020.
| Directorate | URL | Datasets | Change | Type |
|---|---|---|---|---|
| Marine Scotland Data | https://data.marine.gov.scot | 240 $ | +30 | OA |
| Marine Scotland Information | http://marine.gov.scot (infomation tab) | 366 ~ | +66 | O |
| Marine Scotland Maps | https://marinescotland.atkinsgeospatial.com/nmpi/ | 1109 % | +137 | (O) |
($) - 240 datasets with 768 downloadable resources. API to query csv resources (315 resources), as well as DCAT standard inventory on data.marine.gov.scot/data.json
(~) - New this year is 131 marine license pages with access to 2443 license documents. Also around 2,000 links to other organisations, reports etc.
(%) - There are currently 1,109 spatial layers listed. These originate from many different organisations and licenses though.
Source this twitter conversation
The Scottish Government supports Open Procurement. You can download bulk notices or use the API to select data.
You can find the data here
Similarly, you can search the contracts register on this page. Each is OGL licensed.
I tried various ways to find a list of Executive Agencies on the Scottish Goverment website without success. I did find pages such as this guide which had no list. Therefore, turning to Wikipedia I find a list of ten executive agencies (one more than last year). Thanks to Wikipedia's view history feature (why don't public sector websites track and show their changes!?) I can see that two new agencies were added: Forestry and Land Scotland, and Scottish Forestry; and that Historic Environment Scotland has been removed, and made a NDPB (see below).
Where the items in the list below are starred, it indicates that they were previosuly found to have some or all of their open data on the Statistics portal, above. Otherwise, unless flagged below using their initials, I have found no open data for them.
- Accountant in Bankruptcy *
- Disclosure Scotland
- Education Scotland (ES)
- Forestry and Land Scotland (FLS)
- Scottish Forestry (SF)
- Scottish Prison Service (SPS)
- Scottish Public Pensions Agency
- Social Security Scotland (SSS)
- Student Awards Agency for Scotland
- Transport Scotland
ES - In their June 2019 Digital Strategy at Principle 5 says "...Education Scotland’s digital services must: Publish open data where possible to foster innovation in Scottish education...." But further than that I couldn't find specific actions or timescale in relation to open data, nor could I locate any data published since the strategy.
FLS - Querying the FLS site for 'Open Data' was not at all helpful. The second result for Wildfire Prevention said [Mar 2020] that access to it was forbidden. So why list it in public search results? The first result was for "Energy Offering 2019" (whatever that means) and the final line of that page linked to a specific resource on an Arc GIS site offering 10 boundary shapefiles. I found that navigating back to the home of that GIS site (actually a UK-wide Forestry Commission site) then allowed me to locate 19 resources for Scotland. I checked a few. Some had an OGL licence. Others had a 'Custom Licence' such as 'There are no public access constraints to this data. Use of this data is subject to the Open Government LicenceAny maps produced using this data should contain the following Forestry Commission acknowledgement: "Contains, or is based on, information supplied by the Forestry Commission. © Crown copyright and database right 2019 Ordnance Survey [100021242]'.
SF - this has an ARC GIS respoitory with an impressive 71 sets of very intersting data. The several that I looked at were marked as having a "Custom Licence" but on clicking each they stated "In using the data from this website you acknowledge that you have read, understood and agree to abide by the terms and conditions of the Open Government Licence 3.0." No link was provided to the OGL3 to explain its terms, though. They also have a clear, but slightly dated Open Data plan.
SPS - I was able to find a single dataset, an organogram, from 2010 on the Data.gov.uk portal. Nothing else.
The Scottish Public Pensions Agency states "Please note that SPPA ... currently does not publish any open data." At least they are clear unlike Transport Scotland, below.
SSS - Their publication scheme states that 5 star open data is published on the main government statistics data portal (above). Searching there for "Social Security" does turn up five data sets but these are labelled as emanating from SG as a whole rather than from this agency.
Searching on the Student Awards Agency has a site yielded a link to a 2012-2013 statistical publication, but no open data.
Transport Scotland's publication scheme says of open data "Open data made available by the authority as described by the Scottish Government’s Open Data Strategy and Resource Pack, available under an open licence. We comply with the guidance above when publishing data and other information to our website. Details of publications and statistics can be found in the body of this document or on the Publications section of our website." I searched both without success for any OD. Why not say "we don't publish any Open Data"? Also, compare this with the Open Bus Data for England. Story here.
According to Wikipedia in March 2020, the Scottish Government is reponsible for the following 88 Non Departmental Public Bodies, drawn from the more verbose SG Factsheet each of whom should be producing open data according to the Scottish Government's Open Data Strategy.
Where the items in the list below are starred, it indicates that they have some or all of their open data on the Statistics portal, above. Where their initials are in bold there is a para on what I found. Otherwise, I have found no open data for them.
- Food Standards Scotland FSS
- National Records of Scotland *
- Office of the Scottish Charity Regulator OSCR
- Registers of Scotland RoS
- Revenue Scotland *
- Scottish Courts and Tribunals Service SCTS
- Scottish Fiscal Commission
- Scottish Housing Regulator
| Office | URL | Datasets | Change | Type |
|---|---|---|---|---|
| Food Standards Scotland (FSS) | URL | 12 | ? | L |
| Office of the Scottish Charity Regulator (OSCR) | URL | 3 | ? | O |
| Registers of Scotland (RoS) | URL | 4 | ? | O |
| Scottish Courts and Tribunals Service SCTS | URL | 3 | ? | O |
FSS has a new portal with 12 data sets using OGL3 licence.
OSCR allows a download of the [Charity Register data[(https://www.oscr.org.uk/about-charities/search-the-register/charity-register-download), using a OGL3 licence. There are options to download lists of the 300 highest income charities, and of former charities.
RoS publishes four sets of data such as House Price Statistics. It appears to be a mix of commercial and openly licenced data. The copyright page doesn't help to work out which data sets are OGL licenced. Additionally, it should be noted that RoS's INSPIRE Index Polygons were recently released under OGL in place of the previous INSPIRE End User licence.
SCTS produces three sets of reports under Official Statistics which begin as web pages, link to a series of PDFs many of which then have links to heavily-locked down XLSX files. Getting at the data, which at least is OGL3 licenced, is a real challenge.
Where the items in the list below are starred, it indicates that they have some or all of their open data on the Statistics portal, above. Otherwise, I have found no open data for them.
It appears that SEPA, and SNH, in addition to using the Scottish Government's new Statistics Open Data Portal, also publish separately:
| Directorate | URL | Datasets | Change | Type |
|---|---|---|---|---|
| Scottish Environmental Protection Agency (SEPA) | URL | 6 | - | L |
| Scotish Natual Heritage (SNH) | URL | 53 | -1 | O |
Also, I was alerted to the Environment Scotland portal where they have links to other data sources many of which are open, as well as other useful content.
- Accounts Commission for Scotland
- Architecture and Design Scotland
- Bòrd na Gàidhlig
- Cairngorms National Park Authority
- Care Inspectorate *
- Children's Hearings Scotland CHS
- Community Justice Scotland (new)
- Creative Scotland
- Crofting Commission CC
- David MacBrayne Ltd (new) DMBL
- Historic Environment Scotland HES
- Highlands & Islands Airports Ltd (new)
- Highlands & Islands Enterprise
- Independent Living Fund Scotland ILF
- The Loch Lomond and the Trossachs National Park Authority
- National Galleries of Scotland NGS
- National Library of Scotland NLS
- National Museums Scotland NMS
- Police Investigations and Review Commissioner
- Quality Meat Scotland
- Risk Management Authority
- Royal Botanic Garden Edinburgh RBGE
- Scottish Agricultural Wages Board
- Scottish Children's Reporter Administration
- Scottish Criminal Cases Review Commission SCCRC
- Scottish Enterprise SE
- Scottish Environment Protection Agency (SEPA) *
- Scottish Funding Council
- Scottish Futures Trust
- Scottish Land Commission
- Scottish Legal Aid Board
- Scottish Legal Complaints Commission
- Scottish Natural Heritage * (see above)
- Scottish Qualifications Authority SQA
- Scottish Social Services Council SSSC
- Skills Development Scotland
- sportscotland
- VisitScotland *
- Water Industry Commission for Scotland
CC The commission publishes four CSV files under Open Data. There was no clear licence.
CHS in March 2020 had an Open Data Publication Plan from 2017 which stated that they currently publish no OD, but it is due for review in "July 2019." That has vanished - and I can find no publication plan, which is a legal requirement!
HES the HES spatial portal has nine datasets to download. The Terms and Conditions for spatial datasets is now specific that OGL3 is the licence, with the exception of Historic Landuse Assessment Data.
DMBL David MacBrayne Limited is, I believe, is the parent, or operator of Calmac Ferries Limited who, on 1st March 2020 released a new data platform to get data about their 29 ferry routes. This is very welcome. While their intent appears to be to make it Open Data, the website is copyright and there is no specific licensing of the data. After choosing the dates, routes and traffic types you can download a CSV of results.
ILF have a nice "Open Data Communication Plan 2018-2021" which sets out a plan to move from 3- to 5-star data. It is due to be reviewed in Aug 2019 (sic). Sadly, despite the two-year-old assurances I could find no open data on their site (either March or August 2020).
NGS It appears that other than some thinking, in this Jan 2018 paper no data has been released.
NLS Has a plan(2015) and register of open data. Given the impressive breadth of data published on their Data Foundry I suspect that neither of these is up to date.
NMS It appears from their 2019 Model Publication Scheme that they have no open data. _ Incidentally, why do so many insitutions publish the model scheme as their own rather than strip the word model?_
RBGE's Publication Scheme lists three online databases as open data. Only one of these had clear open licensing.
SCCRC's Publication Scheme appears to be misleading. It lists annual reports and research reports under Class 9 Open Data but neither page to which these link appears to have any open data.
SE extol the virtues of OD. In March I noted that they apparently have an events API which doesn't work. This still is no linger functional five months later. They also mention OD in various articles and plans but I could find no published data, which seems extraordinary given the size and range of their operations!
SQA have two datasets on Data.gov.uk but neither are more recent than 2013 data it appears. Their publication schome is unchanged from 2015(!) has no reference to OD. Given the amount of data they posses on school, college and university examinations this is quite incredible. Given the furore over marking of the 2020 School Exams, the need for open data and transparency is all the more pressing, I'd suggest.
SSSC say in their Open Data Publication Plan and Schedule 2016 that they will identify and publish 3 star open data sets by Dec 2016. I could find none.
No open data found (Mar 2020).
- Judicial Appointments Board for Scotland
- Local Government Boundary Commission for Scotland
- Mobility and Access Committee for Scotland
- Scottish Advisory Committee on Distiction Awards
- Scottish Commission on Social Security
- Scottish Law Commission
No open data found (Mar 2020).
- Additional Support Needs Tribunals for Scotland
- First-tier Tax Tribunal for Scotland
- Lands Tribunal for Scotland
- Mental Health Tribunal for Scotland
- Parole Board for Scotland
- Private Rented Housing Panel for Scotland
- Scottish Charity Appeals Panel
- Upper Tax Tribunal for Scotland
No open data found (Mar 2020).
- Caledonian Maritime Assets Lts
- Crown Estate Scotland
- Glasgow Prestwick Airport
- Scottish Canals
- Scottish Water
Where the items in the list below are starred, it indicates that they have some open data on the Statistics portal, above. Otherwise, I have found no open data for them (Mar 2020).
- Accountant in Bankruptcy *
- Disclosure Scotland
- Education Scotland
- Scottish Prison Service
- Scottish Public Pensions Agency
- Social Security Scotland
- Student Awards Agency for Scotland
- Transport Scotland *
Wikipedia also lists Forestry and Land Scotland, and Scottish Forestry but these are dealt with above.
Where the items in the list below are starred, it indicates that they have some open data on the Statistics portal, above. Otherwise, I have found no open data for them (Mar 2020).
- Audit Scotland
- Converor of School Closures Review Panels
- Court of the Lord Lyon
- Drinking Water Quality Regulator
- HM Chief Inspector of Prisons for Scotland
- HM Chief Inspector of Prosecution in Scotland
- HM Fire Service Inspectorate in Scotland (new)
- HM Inspector of Constabulary for Scotland
- Justices of the Peace Advisory Committee (x6)
- Office of the Queen's Printer for Scotland
- Scottish Fire & Rescue Service *
- Scottish Police Authority
- Scottish Road Works Commissioner
If you spot an error - or missing data - please fork this repo and submit a pull request, as four others have kindly done!
Alternatively email me ian@codethecity.org with an update.
Back to the main 2020 review.