Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update demographics.py to use the new UN API format #931

Closed
SeaCelo opened this issue May 16, 2024 · 3 comments
Closed

Update demographics.py to use the new UN API format #931

SeaCelo opened this issue May 16, 2024 · 3 comments

Comments

@SeaCelo
Copy link

SeaCelo commented May 16, 2024

The API for the UN population data used in demographics.py is being updated to use bearer tokens. The system is still not live, but the new format will look like this (requires a value for your_token_here):

-- USING PYTHON HTTP.CLIENT
import http.client
conn = http.client.HTTPSConnection("population.un.org")
payload = ''
headers = {
  'Authorization': 'Bearer your_token_here'
}
conn.request("GET", "/dataportalapi/api/v1/data/indicators/1/locations/688/start/2001/end/2005?pagingInHeader=false&format=json", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))
--
OR
--- USING PYTHON REQUESTS ---
import requests
url = https://population.un.org/dataportalapi/api/v1/data/indicators/1/locations/688/start/2001/end/2005?pagingInHeader=false&format=json
payload = {}
headers = {
  'Authorization': 'Bearer your_token_here'
}
response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)
---

As soon as the API is live, and the method to generate the bearer token is published, I will update and test the code.

@jdebacker
Copy link
Member

Great - thanks for keeping up with this @SeaCelo !

@jdebacker
Copy link
Member

@rickecon and @SeaCelo Here's an idea: Since the UN data files are quite small (150KB gets you almost 100 years of the population distribution by age) and since relatively few users will be able to get a UN API token, what if we create repo in EAPD-DRB calls "UN Population Data". This will have a README briefly explaining the files and then directories for each OG-XXX model, which contain 5 CSV files with the relevant data for each country calibration.

We can then change the ogcore.demographics.get_un_data() to have its current functionality if someone has the API token, but if not, it reads the relevant data files from the "UN Population Data" repo.

Some advantages of this:

  1. It centralizes the storage of data so when there are changes to the UN WPP data (e.g., they add another forecast year), there's just one place to update them.
  2. The ogcore.demographics can remain largely the same.

Let me know your thoughts.

@jdebacker
Copy link
Member

Addressed in PR #936 using the methods suggested in the previous comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants