This site can be found at https://www.rtei.org and it is powered by Wagtail
-
Create a Postgres User and Database (password
pass):sudo -u postgres createuser -d -S -R -P -l rtei sudo -u postgres createdb -O rtei rtei -E utf-8 -
Create a Virtualenv and clone this repo:
mkvirtualenv rtei mkdir src && cd src git clone git@github.com:okfn/rtei.git -
Install requirements:
cd rtei pip install -r requirements.txt -
Set up the database:
python manage.py migrate -
Create a super user and run the site:
python manage.py createsuperuser python manage.py runserver
RTEI requires the following values to be set as env vars:
SECRET_KEY='xxx'
# Email to receive contact requests from the form on /about/contact-us/
RTEI_CONTACT_FORM_EMAIL='contact_form@example.com'
EMAIL_HOST='xxx'
EMAIL_HOST_PASSWORD='xxx'
EMAIL_HOST_USER='xxx'
# S3 Storage for docs and resources
AWS_ACCESS_KEY_ID='xxx'
AWS_SECRET_ACCESS_KEY='xxx'
AWS_STORAGE_BUCKET_NAME='xxx'
# Postgres DB to use (dev defaults to postgres://rtei:pass@localhost/rtei)
DATABASE_URL='postgres:...'
# ElasticSearch endpoint use for search (dev defaults to http://localhost:9200)
ELASTICSEARCH_URL='http://example.com/es'
# Google Analytics site code for production
GOOGLE_ANALYTICS_CODE='xxx'
Install the tests requirements:
pip install -r test-requirements.txt
Run:
./manage.py test
Use the staging database's dump to populate contents for development. Copy images from the staging site to mimic the blog appearance.
The JSON data needed to power the visualizations on the site is built using the build_data.py script, which parses the original Excel (xlsx) file located at data.
Before running the script you must install its requirements:
cd rtei
pip install -r data-requirements.txt
Run ./build_data.py -h to see all the options available.
Most of the times you will want to:
- Update
/rtei/static/data/rtei_data_{year}.xlsxif necessary - Run
./build_data.py all
The JSON data files are generated in rtei/static/data/{year} by default. These files are:
-
indicators.json: Master dictionary that links every indicator code to its title (and level). Indicators are nested, eg:[ { "code": "1", "children": [ { "code": "1.1", "children": [ { "code": "1.1.1", "children": [ { "code": "1.1.1a", "level": 4, "title": "The International Covenant of Economic, Social and Cultural Rights" }, { "code": "1.1.1b", "level": 4, "title": "The Convention on the Rights of the Child" }, ... ] } } } ... ] -
themes.json: Master dictionary that links every theme code to its title (and level). Themes are nested, eg:[ { "code": "1", "level": 1, "title": "Children with Disabilities" "children": [ { "code": "1.A.A", "level": 2, "title": "Structure and Support" } ] }, ... ] -
scores_per_country.json: Contains all level 1 and 2 scores for all countries, as well as the overall one:{ "CL": { "1": 64.76, "1.1": 100 "1.2": 50 "1.3": 0 "1.4": 84.4, "1.6": 89.4, "2": 51.95, "2.1": 33.3, "2.2": 96 "2.3": 73.2, "2.4": 5.3, "3": 88.3833, "3.1": 89.6, "3.2": 83.25, "3.3": 92.3, "4": 80.8, "4.1": 68.4, "4.2": 77.7, "4.3": 96.3, "5": 55.4333, "5.1": 66.5, "5.2": 66.5, "5.3": 33.3, "index": 68.2653 }, "NG": { "1": 94.56, "1.1": 100 "1.2": 100 "1.3": 100 "1.4": 84.4, "1.6": 88.4, "2": 53.275, "2.1": 25.6, "2.2": 50 "2.3": 37.5, "2.4": 100 "3": 77.8333, "3.1": 97.4, "3.2": 58.3, "3.3": 77.8, "4": 79.9667, "4.1": 100 "4.2": 44.3, "4.3": 95.6, "5": 75.0667, "5.1": 66.5, "5.2": 92.7, "5.3": 66 "index": 76.1403 }, ... -
c3_scores_per_country.json: C3 optimized data for the country bar charts. Contains all level 1 and 2 scores normalized for all countries, as well as the overall one and the transversal themes:[ { "1": 12.85, "1.1": 20 "1.2": 10 "1.3": 0 "1.4": 16.88, "1.5": 17.39, "2": 15.13, "2.1": 8.32, "2.2": 24 "2.3": 18.3, "2.4": 25 "index": 72. "name": "Chile", "t1A.A": 68.4, "t2A": 1 "t3A": 83.07, "t3B": 72.03, "t4A": 55.53, "t5A": 76.14, "t6A": 71.1, "t7A": 75, "t7B": 75.59, "t8A": 66.66, "t9A": 0 }, ... ] -
{country_code}.json(egCL.json): For each of the countries available, contains all values for all indicators for that particular country. The values are the user-friendly responses shown on the frontend:{ "1": 64.267, "1.1": 100 "1.1.1a": "Yes", "1.1.1b": "Yes", "1.1.1c": "Yes", "1.1.1d": "Yes", "1.1.1e": "Yes", "1.1.1f": "Yes", "1.1.1g": "Yes", "1.1.1h": "Yes", "1.1.2a": "Yes", "1.1.3a": "Yes", "1.1.3b": "Yes", "1.1.3c": "Yes", "1.1.4a": "Yes", "1.1.4b": "Yes", "1.1.4c": "Yes", "1.1.4d": "Yes", "1.1.5a": "Not Applicable", "1.1.5b": "Not Applicable", ...
The map visualizations are powered by a TopoJSON file. This is built using two input files, data/countries.geojson, which is constant and does not need to be updated, and data/scores_per_country.csv, which needs to be updated:
./build_data.py -o data scores-per-country-csv
Install the topojson command (use a version lower than 2.0):
npm install topojson@1.6.27
This is the full command that needs to be done (note that the output file goes to the rtei/static/data):
topojson -p -o rtei/static/data/2021/countries.topojson --stitch-poles false --id-property iso2 -e data/scores_per_country.csv data/countries.geojson
Dump a new set of site data (this is the internal data for managing the site, like pages etc, not the data powering the visualizations):
cd rtei
python manage.py dumpdata --natural-foreign --indent=4 -e contenttypes -e auth.Permission -e sessions -e wagtailcore.pagerevision -e auth.user > rtei/fixtures/data.json
The site is hosted on Heroku:
-
rteiis the staging site, which is deployed automatically from Travis on each push to master. -
rtei-productionis the production site, deployed manually via git remote:heroku git:remote -r production -a rtei-production git push production master
Database migrates can be run on heroku against the production settings with:
heroku run python ./manage.py migrate --settings=rtei.settings.production
Translations are managed on Transifex. You will need to install the Transifex command line client:
pip install transifex-client
If you haven't already done it, you need to create a ~/.transifexrc file with the following contents:
[https://www.transifex.com]
hostname = https://www.transifex.com
username = YOUR_USERNAME
password = YOUR_PASSWORD
token =
To test that it's properly configured, run the following on the repo directory:
tx status
You should see something along these lines:
rtei -> rtei (1 of 1)
Translation Files:
- en: locale/django.pot (source)
- ar: locale/ar/LC_MESSAGES/django.po
- es: locale/es/LC_MESSAGES/django.po
- fr: locale/fr/LC_MESSAGES/django.po
New strings added to the source code that need to be translated must be regularly extracted and uploaded to Transifex.
Note: This also includes strings in the original data! (eg Indicator titles or responses)
To do so run the following:
# Extract translatable strings from data
./build_data.py translation-strings
# Extract strings from source code into po files (also keep the master pot file)
./manage.py makemessages --keep-pot
# Upload to Transifex
tx push -s -t --skip
#Commit new po files
git commit -am "Update translation files with new strings"
At this point the strings are available for translation on Transifex.
Once the translators have finished workin on Transifex, update the source code translations with the following commands:
# Pull strings from Transifex
tx pull
# Compile strings catalogue, ie the locale/*.mo files (You need to restart the server after this)
./manage compilemessages
# Commit the changes
git commit -am "Updated strings from Transifex"
The project uses SASS. To install it run:
sudo gem install sass
To rebuild the CSS, watching for changes in the scss files, run:
sass rtei/static/scss/rtei.scss:rtei/static/css/rtei.css --watch