home
qed exposes science apps via a public server and a server that internally faces the EPA.
- qed public server via aws (qeduser: ecoDomain2019) - https://qedkube.edap-cluster.com/
- qed public server via azure (qeduser: ecoDomain2019) - https://qed.epacdx.net/
- qed dev public server via azure (qeduser: ecoDomain2019) - https://qeddev.epacdx.net/
- qed blue dev server via aws (qeduser: ecoDomain2019) - https://qedblue.edap-cluster.com/
- qed green dev server via aws (qeduser: ecoDomain2019) - https://qedgreen.edap-cluster.com/
These federal servers use the Trusted Internet Connection (TIC) reference architecture. The parent repository for this project is hosted at the USEPA github repository; this repo is a submodule to the USEPA QED github repo.
qed contains the source code for a number of EPA software applications that are integrated into a web deployment. The qed repository is the most critical repository in this deployment. It contains over 30 git submodules that control the networking setup, the container deployment, the front end web applications, the back end science services deployed within our virtual machine environment as well as connections to a number of other publicly available services. Details on how to deploy qed on the server and locally for development are provided on the dev workflows link.
The docker environment is deployed on virtual machines hosted by cgifederal. We are currently running 4 Red Hat Enterprise Linux 7.5 and 4 Windows Server 2012 instances. QED is deployed on 2 of the red hat servers (one public and one limited to the EPA intranet). Updates are done using the methods described in the dev workflows wiki page on this site. Deployment is also partially automated through the use of private ansible repository scripts.
The docker deployment is orchestrated by the docker-compose.yml file that is located in the qed repository. This file kicks off a series of debian linux-based docker image builds/downloads to create the different containers (currently 14), docker data volumes and the network connections on the qed server. In addition, these services can connect to other servers on the CGI, EPA and public networks. These containers comprise the major elements of the qed technology stack and are summarized in this table that has a mix of official docker images (nginx, python, node, redis) and docker images we manage/build.
| purpose | repo | branches | dockerfile | image | builds |
|---|---|---|---|---|---|
| nginx official | nginx | NA | NA | nginx | NA |
| nginx | qed_nginx | qed_nginx | qed_nginx | qed_nginx | qed_nginx |
| python official | python | NA | NA | python | NA |
| python | qed_py3 | qed_py3 | qed_py3 | qed_py3 | qed_py3 |
| python packages | requirements_qed | requirements_qed | requirements_qed | NA | NA |
| django | qed | qed | qed | qed | qed |
| flask | flask_qed | flask_qed | flask_qed | flask_qed | qed |
| node official | node | NA | NA | node | NA |
| node | node_qed | node_qed | node_qed | node_qed | node_qed |
| redis official | redis | NA | NA | redis | NA |
| celery | celery_qed | celery_qed | celery_qed | celery_qed | celery_qed |
Once the build process is complete, the qed docker file initiates the docker_start.sh shell script which runs uwsgi with the uwsgi.ini configuration file and the wsgi_docker.py file. This process starts uwsgi, which serves as the http router that binds python wsgi to our web server -- nginx.
Once the docker environment is created on the server, requests are handled by nginx and routed to either django or flask depending on the call. The nginx configuration file specifies how this traffic is initially routed when requests hit the server.
The qed repository contains the main django web application and a number of science-based apps. The sequence of python code and service that run in django include
| repo | branches | travis | codacy | code climate | landscape |
|---|---|---|---|---|---|
| qed | qed |  |
 |
 |
|
| static_qed | static_qed | NA |  |
NA | |
| templates_qed | templates_qed | NA |  |
NA | |
| qed_cts | qed_cts | NA |  |
 |
|
| qed_cyan | qed_cyan | NA |  |
|
|
| qed_hem | qed_hem | NA |  |
|
|
| qed_hms | qed_hms | NA |  |
 |
|
| qed_hwbi | qed_hwbi | NA |  |
 |
|
| qed_pisces | qed_pisces | NA |  |
 |
|
| qed_splash | qed_splash | NA |  |
 |
|
| qed_pram | qed_pram | NA |  |
 |
|
| qed_wqt | qed_wqt | NA | NA |  |
 |
The django apps represent the different science application web-based front ends. These are pluggable submodules that get added to the host django project (qed or qed_*). They support EPA regulatory needs across three research programs: css, shc and sswr. They dynamically build the web pages using the EPA drupal style; leverage html templates, cascading style sheets, static images, etc. They also make RESTful calls to the back end science models and receive the model output so that the results can be rendered in a browser with javascript and python based visualization tools.
| repo | branches | travis | codacy | code climate | landscape |
|---|---|---|---|---|---|
| cts_app | cts_app | NA |  |
 |
|
| cyan_app | cyan_app | NA |  |
 |
|
| hem_app | hem_app | NA |  |
 |
|
| hms_app | hms_app | NA |  |
 |
|
| hwbi_app | hwbi_app | NA |  |
 |
|
| pisces_app | pisces_app | NA |  |
 |
|
| pram_app | pram_app | NA |  |
 |
|
| splash_app | splash_app | NA |  |
 |
|
| wqt_app | wqt_app | NA |  |
 |
Some of the science capabilities are embedded in the above django apps, but for more substantial calculations, especially those that are going to be exposed as a REST API, they are deployed as a separate repository and exposed via flask.
| repo | branches | travis | codacy | code climate |
|---|---|---|---|---|
| cts_calcs | cts_calcs | NA |  |
|
| hms_backend | hms_backend | NA |  |
|
| pram_qaqc_reports | pram_qaqc_reports | NA |  |
|
| sam2 | sam2 | NA |  |
|
| ubertool | ubertool | NA |  |
We leverage a number of middleware and back end technologies so that we can run science models in their native environments. These components include celery (python-based task queue), flask (python-based API framework), node.js (server-side JavaScript runtime), opencpu (APIs for R scripts) and our own python-based testing repositories. We are using flask for all our data- and model- based restful endpoints and proxy them to the outside using our django front end.
| repo | branches | travis | codacy | code climate |
|---|---|---|---|---|
| cts_api | cts_api | NA |  |
|
| cts_celery | cts_celery | NA |  |
|
| cts_channels | cts_channels | NA |  |
|
| cts_nodejs | cts_nodejs | NA |  |
|
| cts_stress | cts_stress | NA | NA |  |
| cts_testing | cts_testing | NA |  |
|
| ctsws | ctsws | NA |  |
|
| flask_qed | flask_qed | NA |  |
|
| hms_flask | hms_flask | NA |  |
|
| pram_flask | pram_flask | NA |  |
|
| qed_smoketests | qed_smoketests | NA | NA |  |
| temp_config | temp_config | NA | NA |  |
| varroapopserver | varroapopserver | NA |  |