DesignSafe-CI Portal

Prequisites for running the portal application

The DesignSafe-CI Portal can be run using Docker and Docker Compose. You will need both Docker and Docker Compose pre-installed on the system you wish to run the portal on.

• Install Docker
• Install Docker Compose
• Install Make
• Node.js 20.x

If you are on a Mac or a Windows machine, the recommended method is to install Docker Desktop, which will install both Docker and Docker Compose, which is required to run Docker on Mac/Windows hosts.

First time setup

1. Clone the repo

   $ git clone https://github.com/DesignSafe-CI/portal.git
   $ cd portal
   

2. Configure environment variables

   Make a copy of designsafe.sample.env and rename it to designsafe.env. Configure variables as necessary. See designsafe.sample.env for details of configuration.

   Required variables:

   • DJANGO_DEBUG: should be set to True for development
   • DJANGO_SECRET_KEY: should be changed for production
   • TAS_*: should be set to enable direct access to django.contrib.admin
   • TAPIS_*: should be set to enable Tapis API integration (authentication, etc.)
   • RT_*: should be set to enable ticketing

   Make a copy of rabbitmq.sample.env then rename it to rabbitmq.env.

3. Configure ngrok

   a. Install ngrok, and create an ngrok account.

   b. Copy conf/env_files/ngrok.sample.env to conf/env_files/ngrok.env.

   c. In conf/env_files/ngrok.env, set the NGROK_AUTHTOKEN and NGROK_DOMAIN variables using your authtoken and static ngrok domain found in your ngrok dashboard.

4. Build the containers and frontend packages

   1. Containers:

      make build-dev

      or

      docker-compose -f conf/docker/docker-compose-dev.yml build

   2. Angular Frontend + static assets:

      npm ci
      docker run -v `pwd`:`pwd` -w `pwd` -it node:16  /bin/bash -c "npm run build"

      Note: If you are working with the frontend code and want it to automatically update, use npm run dev rather than npm run build to have it build upon saving the file.

   3. React Frontend (in another terminal):

      cd client
      npm ci
      npm run start

5. Start local containers

   $ make start
   

   or

   $ docker-compose -f ./conf/docker/docker-compose-dev.all.debug.yml up
   

   then, in a new tab

   $ docker exec -it des_django bash
   $ ./manage.py migrate
   $ ./manage.py collectstatic --ignore demo --no-input
   $ ./manage.py createsuperuser
   

6. Setup local access to the portal:

   Add a record to your local hosts file for 127.0.0.1 designsafe.dev

   $ sudo vim /etc/hosts
   

   Now you can navigate to designsafe.dev in your browser.

Next steps

Installing local CA

Every file needed is in conf/nginx/certs.

OSX

1. Open mac's Keychain Access
2. File > Import Items
3. Navigate to $PROJECT/conf/nginx/certificates
4. Select designsafe.dev.crt
5. Search for designsafe and double click on Designsafe.dev
6. Click on Trust and select "Trust Always"
7. Close the window to save.

Linux

Ubuntu & Debian

1. $ cd $PROJECT/conf/nginx/certificates
2. $ sudo mkdir /usr/local/share/ca-certificates/extra
3. $ sudo cp ca.pem /usr/local/share/ca-certificates/extra/designsafeCA.pem
4. $ sudo update-ca-certificates

Fedora & CentOS & RHEL

1. $ sudo dnf install ca-certificates or
   $ sudo yum install ca-certificates
2. $ sudo update-ca-trust enable
3. $ cd $PROJECT/conf/nginx/certificates
4. $ sudo cp ca.pem /usr/share/pki/ca-trust-source/anchors/designsafeCA.pem or
   $ sudo cp ca.pem /etc/pki/ca-trust/source/anchors/designsafeCA.pem
5. $ sudo update-ca-trust

Firefox UI

1. Go to preferences
2. Search for Authorities
3. Click on "View Certificates" under "Certificates"
4. On the Certificate Manager go to the "Authorities" tab
5. Click on "Import..."
6. Browse to $PROJECT/conf/nginx/certificates
7. Select ca.pem

Firefox CLI (not tested)

1. sudo apt-get install libnss3-tools (or proper package manager)
2. certutil -A -n "designsafeCA" -t "TCu,Cu,Tu" -i ca.pem -d ${DBDIR}
3. $DBDIR differs from browser to browser for more info: Chromium: https://chromium.googlesource.com/chromium/src/+/master/docs/linux_cert_management.md Firefox: https://support.mozilla.org/en-US/kb/profiles-where-firefox-stores-user-data?redirectlocale=en-US&redirectslug=Profiles#How_to_find_your_profile

NOT REQUIRED: Creating Local CA and signed cert

1. Generate RSA-2048 key for CA: openssl genrsa -des3 -out ca.key 2048 (This file should already be in the repo)
2. Generate root CA certificate: openssl req -x509 -new -nodes -key ca.key -sha256 -days 365 -out ca.pem (Root CA cert is valid for 365 days. Keep any form values to "Designsafe CA")
3. Generate RSA-2048 key for local dev site: openssl genrsa out designsafe.dev.key 2048 (This file should already be in the repo)
4. Generate Cert Request (CSR): openssql req -new -key -designsafe.dev.key -out designsafe.dev.csr (Keep any form values to "Designsafe CA")
5. Make sure designsafe.dev.ext is correct
6. Generate Cert: openssl x509 -req -in designsafe.dev.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out designsafe.dev.crt -days 365 -sha256 -extfile designsafe.dev.ext (Cert is valid for 365 days. Keep default form values defined in .conf file)
7. Files created: designsafe.dev.key (site private key), designsafe.dev.csr (site certificate signing request), designsafe.dev.crt (actual site certificate), ca.key (CA private key) and ca.pem (CA certificate).

Re-creating self signed cert

The necessary configuration is already in conf/nginx/designsafe.dev.conf.

Run this command to create a self signed cert using the corresponding configuration.

openssl req -config conf/nginx/designsafe.dev.conf -new -sha256 -newkey rsa:2048 -nodes -keyout conf/nginx/designsafe.dev.key -x509 -days 365 -out designsafe.dev.crt

Developing DesignSafe-CI Portal

Apps

DesignSafe custom apps should be put into designsafe/apps. You can then enable them in the Django settings.py with designsafe.apps.{app_name}.

CSS/Styling

See the DesignSafe Styles Reference for style reference and custom CSS documentation.

Updating Python dependencies

This project uses Python Poetry to manage dependencies. To add a new dependency:

1. Run poetry add $NEW_DEPENDENCY.
2. Rebuild the dev image with make build-dev

Testing

The easiest way to run the tests is from inside a running Docker container. While you can install all the Python/Django/npm dependencies locally (ideally within a virtualenv), this is already done in the docker container.

We assume you have the image built or checked out locally and it is called des_django.

Django tests

Django tests should be written according to standard Django testing procedures.

You can run Django tests with the following command:

$ docker exec -it des_django pytest -ra designsafe

Frontend tests

Frontend tests are Vitest tests executed using Nx.

To run frontend tests, run the command:

$ npm run test

Development setup

Use docker compose to run the portal in development. The default compose file, docker-compose.yml runs the main django server in development mode with a redis service for websockets support. You can optionally enable the EF sites for testing.

$ make build-dev
$ make start
$ npm run start
$ docker run -v `pwd`:`pwd` -w `pwd` -it node:16  /bin/bash -c "npm run dev"

When using this compose file, your Tapis Client should be configured with a callback_url of http://$DOCKER_HOST_IP:8000/auth/tapis/callback/.

$ docker-compose -f docker-compose-http.yml build
$ docker-compose -f docker-compose-http.yml up

Production setup

Production deployment is managed by Camino. See https://github.com/TACC/Camino.