Feat/1544 add nginx configuration increase security performance #2173

raftmsohani · 2022-10-03T12:30:45Z

Summary of Changes

The Nginx CSP and server configuration need to be revised and correctly documented for both local and Cloud.gov deployment to increase security.

Pull request closes #1544 , #1543 and #2031

How to Test

source commands.sh
tdrs-start

It is important to use commands.sh to start the application since local deployment needs external network to be created first to connect the backend and frontend docker-compose configs.

Deliverables

More details on how deliverables herein are assessed included here.

Nginx configuration for local and cloud.gov
Added documentation and comments for Nginx configs
Gunicorn configuration and documentation
Separated local and cloud.gov configurations for both Nginx and Gunicorn

Deliverable 1: Accepted Features

Checklist of ACs:

[insert ACs here]
lfrohlich and/or adpennington confirmed that ACs are met.

Deliverable 2: Tested Code

Are all areas of code introduced in this PR meaningfully tested?
- If this PR introduces backend code changes, are they meaningfully tested?
- If this PR introduces frontend code changes, are they meaningfully tested?
Are code coverage minimums met?
- Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
- Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

Are backend code style checks passing on CircleCI?
Are frontend code style checks passing on CircleCI?
Are code maintainability principles being followed?

Deliverable 4: Accessible

Does this PR complete the epic?
Are links included to any other gov-approved PRs associated with epic?
Does PR include documentation for Raft's a11y review?
Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

Does this PR provide background for why coding decisions were made?
If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
If this PR introduces dependencies, are their licenses documented?
Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

Does the OWASP Scan pass on CircleCI?
Do manual code review and manual testing detect any new security issues?
If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

the purpose of the research
methods used to conduct the research
who participated in the research
what was tested and how
impact of research on TDP
(if applicable) final design mockups produced for TDP development

codecov · 2022-11-27T18:49:47Z

Codecov Report

Merging #2173 (abd8226) into develop (8b9dc4e) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #2173   +/-   ##
========================================
  Coverage    94.13%   94.13%           
========================================
  Files           96       96           
  Lines         2846     2846           
  Branches       263      263           
========================================
  Hits          2679     2679           
  Misses         120      120           
  Partials        47       47

Flag	Coverage Δ
dev-backend	`94.14% <100.00%> (ø)`
dev-frontend	`94.11% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
tdrs-backend/tdpservice/settings/common.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b9dc4e...abd8226. Read the comment docs.

…e-security-performance

tdrs-frontend/nginx/README.md

andrew-jameson · 2023-01-04T19:45:04Z

tdrs-frontend/nginx/README.md

+
+In general for TDRS, the *Nginx* handles incoming HTTP requests to both frontend and backend. The frontend requests are handled directly while the backend requests are forwarded to the backend server.
+
+![Cloud.gov Architecture](./src/arch1.jpg)


We might be able to keep a single diagram and just update this one:

https://github.com/raft-tech/TANF-app/blob/develop/docs/Security-Compliance/boundary-diagram.md

andrew-jameson · 2023-01-06T19:02:30Z

@raftmsohani Can you verify that the ip whitelist allows the territories (Puerto Rico, Guam, Virgin Islands)? Credit to @ADPennington

raftmsohani · 2023-01-09T14:21:33Z

tdrs-backend/docker-compose.yml

@@ -93,7 +93,7 @@ services:
      bash -c "./wait_for_services.sh &&
               ./gunicorn_start.sh && celery -A tdpservice.settings worker -l info"
    ports:
-      - "8080:8080"
+      #- "8080:8080"


Port assignment should be removed -> delete #- "8080:8080"

andrew-jameson · 2023-01-09T17:30:09Z

tdrs-frontend/nginx/local/locations.conf

+location = /profile {
+    index index.html index.htm;
+    try_files $uri $uri/ /index.html;
+}
+
+location = /home {
+    index index.html index.htm;
+    try_files $uri $uri/ /index.html;
+}
+
+location ^~ /data-files {
+    index index.html index.htm;
+    try_files $uri $uri/ /index.html;
+}
+
+location = /login {
+    index index.html index.htm;
+    try_files $uri $uri/ /index.html;
+}
+
+location = / {
+    index index.html index.htm;
+    try_files $uri $uri/ /index.html;
+}


Can these locations be collapsed into one block? They appear identical.

EDIT: Looks like this was the same as from prior locations.conf. It's not a real change but didn't notice initially since we don't have before/after diff on several files.

it can be done for sure. An easy fix

The before after missing is because of file relocation -> moved some of the files to another directory

The before after missing is because of file relocation -> moved some of the files to another directory

Did you just manually move it vs using git mv? Minor nit

Virgin Islands

I bet the test will fail because I will have to add the following lists:

https://lite.ip2location.com/virgin-islands-(u.s.)-ip-address-ranges https://lite.ip2location.com/guam-ip-address-ranges?lang=en_US

Virgin Islands

I bet the test will fail because I will have to add the following lists:

https://lite.ip2location.com/virgin-islands-(u.s.)-ip-address-ranges https://lite.ip2location.com/guam-ip-address-ranges?lang=en_US

Added both location to whitelist

andrew-jameson · 2023-01-09T17:36:21Z

tdrs-frontend/nginx/cloud.gov/buildpack.nginx.conf

@@ -0,0 +1,81 @@
+


Before merging, I would want to deploy-to/test-in prod and would want to coordinate this with @ADPennington and setup maintenance window w/ UX

tdrs-frontend/nginx/local/default.conf.template

andrew-jameson · 2023-01-09T17:38:29Z

tdrs-frontend/nginx/local/default.conf.template

+        # prevent MIME sniffing
+        add_header X-Content-Type-Options: nosniff;
+
+        # CSP header options. All options are set either to none or self except


Looks like the second half of this sentence got missed

andrew-jameson · 2023-01-09T17:39:42Z

tdrs-frontend/nginx/local/locations.conf

+#location ~ \.(css)$ {
+#    add_header  Content-Type    text/css;
+#}
+
+#location = /nginx_status {
+#    stub_status on;
+#    access_log   off;
+#    deny all;
+#}
+


Suggested change

#location ~ \.(css)$ {

# add_header Content-Type text/css;

#}

#location = /nginx_status {

# stub_status on;

# access_log off;

# deny all;

#}

kept the /nginx_status for local.

andrew-jameson · 2023-01-09T17:41:03Z

tdrs-frontend/nginx/local/default.conf.template

+        set $CSP "${CSP}media-src 'none';";
+        set $CSP "${CSP}prefetch-src 'none';";
+        set $CSP "${CSP}form-action 'none';";
+        set $CSP "${CSP}script-src-elem 'self' http://localhost:* http://www.w3.org;";


Suggested change

set $CSP "${CSP}script-src-elem 'self' http://localhost:* http://www.w3.org;";

set $CSP "${CSP}script-src-elem 'self' http://localhost:*;";

Are we sure we want the w3 part in there? Confused on why. Are we pulling JS/CSS?

I believe the icons needed w3. In any case, this should be safe.

removed as suggested!

tdrs-frontend/nginx/cloud.gov/buildpack.nginx.conf

andrew-jameson · 2023-01-09T17:43:08Z

tdrs-frontend/nginx/cloud.gov/buildpack.nginx.conf

+                           '"http_user_agent": $http_user_agent, '
+                           '"cookies=$http_cookie;"  "server=$server_name" "http_host=$http_host"';
+
+    limit_req_zone $binary_remote_addr zone=limitreqsbyaddr:20m rate=10r/s;


I think I commented on this elsewhere already but I think we might need to stress test or up the rate-limit here.

We discussed this briefly and I think we should have to increase the limit. I have done some testing and got a rejection only once. But one caveat is: when frontend auth check function gets rejected, if user is already logged in, the backend will log them out.

Increased the rate! Resolved

andrew-jameson · 2023-01-09T17:44:35Z

tdrs-frontend/nginx/cloud.gov/locations.conf

+#location ~ \.(css)$ {
+#    add_header  Content-Type    text/css;
+#}
+
+#location = /nginx_status {
+#    stub_status on;
+#    access_log   off;
+#    deny all;
+#}
+


Suggested change

#location ~ \.(css)$ {

# add_header Content-Type text/css;

#}

#location = /nginx_status {

# stub_status on;

# access_log off;

# deny all;

#}

raftmsohani · 2023-01-09T17:48:41Z

@raftmsohani Can you verify that the ip whitelist allows the territories (Puerto Rico, Guam, Virgin Islands)? Credit to @ADPennington

I will see what can be done here! will update this post once done!

raftmsohani · 2023-01-09T18:47:54Z

scripts/deploy-backend.sh

@@ -117,6 +117,7 @@ update_backend()
        cf map-route tdp-backend-prod api-tanfdata.acf.hhs.gov


Add map-route to prod

raftmsohani · 2023-01-10T15:25:02Z

tdrs-frontend/nginx/cloud.gov/locations.conf

+#    deny all;
+#}
+
+location ^~ /v1/ {


Have to add /admin location

andrew-jameson · 2023-01-10T16:45:34Z

@raftmsohani Can you verify that the ip whitelist allows the territories (Puerto Rico, Guam, Virgin Islands)? Credit to @ADPennington

I will see what can be done here! will update this post once done!

UX might be able to coordinate tests with Virgin Islands or Guam to test against develop/staging environment.

cc @ADPennington

raftmsohani · 2023-01-10T16:48:53Z

Virgin Islands

I bet the test will fail because I will have to add the following lists:

https://lite.ip2location.com/virgin-islands-(u.s.)-ip-address-ranges
https://lite.ip2location.com/guam-ip-address-ranges?lang=en_US

raftmsohani · 2023-01-10T18:21:13Z

tdrs-frontend/nginx/mime.types

@@ -1,10 +1,12 @@
 types {
  text/html html htm shtml;
  text/css css;
+  text/css chunk.css;


This is really not a requirement, but I thought it is better to add to save us from future problem.

Co-authored-by: Andrew <84722778+andrew-jameson@users.noreply.github.com>

andrew-jameson · 2023-04-10T19:55:04Z

Out of date per @raftmsohani. Please see #2449

raftmsohani added 3 commits September 23, 2022 13:24

local containerized nginx

d9ce5b0

local containerized nginx + Nginx

553ff80

corrected the headers for local env

20488fd

raftmsohani self-assigned this Oct 28, 2022

raftmsohani added the backend label Oct 28, 2022

raftmsohani added 3 commits November 25, 2022 12:58

Changes required for local docker-nginx

c8dc4ce

required changes for circleci env

399f60f

linting backend

1f0f714

raftmsohani and others added 10 commits November 28, 2022 13:29

Delete .env.

ed7b4c3

merge develop

1ebc83f

Nginx changes for deployment

1099b05

add create network to the commands

62f6eaa

linting

3dd740e

Merge branch 'develop' into feat/1544-Add-Nginx-configuration-increas…

5fca87d

…e-security-performance

Corrected .circleci commands/jobs

272de5c

added create network for all jobs

5e1575b

added create network to the correct command.yml

1bfc54f

removed duplicate create network

8618031

raftmsohani added Deploy with CircleCI-raft Deploy to https://tdp-frontend-raft.app.cloud.gov through CircleCI and removed Deploy with CircleCI-raft Deploy to https://tdp-frontend-raft.app.cloud.gov through CircleCI labels Dec 6, 2022

Nginx changes for proxypass backend from Nginx

3dad9f0

raftmsohani removed the Deploy with CircleCI-raft Deploy to https://tdp-frontend-raft.app.cloud.gov through CircleCI label Dec 9, 2022

andrew-jameson reviewed Jan 4, 2023

View reviewed changes

tdrs-frontend/nginx/README.md Outdated Show resolved Hide resolved

andrew-jameson reviewed Jan 4, 2023

View reviewed changes

andrew-jameson added Deploy with CircleCI-raft Deploy to https://tdp-frontend-raft.app.cloud.gov through CircleCI and removed Deploy with CircleCI-raft Deploy to https://tdp-frontend-raft.app.cloud.gov through CircleCI labels Jan 4, 2023

raftmsohani commented Jan 9, 2023

View reviewed changes

andrew-jameson reviewed Jan 9, 2023

View reviewed changes

tdrs-frontend/nginx/local/default.conf.template Show resolved Hide resolved

andrew-jameson reviewed Jan 9, 2023

View reviewed changes

tdrs-frontend/nginx/cloud.gov/buildpack.nginx.conf Show resolved Hide resolved

andrew-jameson reviewed Jan 9, 2023

View reviewed changes

raftmsohani commented Jan 9, 2023

View reviewed changes

raftmsohani commented Jan 10, 2023

View reviewed changes

tdrs-frontend/nginx/cloud.gov/locations.conf

# deny all;

#}

location ^~ /v1/ {

Copy link

Author

raftmsohani Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have to add /admin location

raftmsohani commented Jan 10, 2023

View reviewed changes

raftmsohani changed the title ~~[WIP] Feat/1544 add nginx configuration increase security performance~~ Feat/1544 add nginx configuration increase security performance Jan 11, 2023

raftmsohani and others added 4 commits January 25, 2023 08:21

Update tdrs-frontend/nginx/README.md

81a7c5d

Co-authored-by: Andrew <84722778+andrew-jameson@users.noreply.github.com>

Update tdrs-frontend/nginx/README.md

d7a2175

Co-authored-by: Andrew <84722778+andrew-jameson@users.noreply.github.com>

Update tdrs-frontend/nginx/README.md

fb17384

Co-authored-by: Andrew <84722778+andrew-jameson@users.noreply.github.com>

Update tdrs-frontend/nginx/README.md

abd8226

Co-authored-by: Andrew <84722778+andrew-jameson@users.noreply.github.com>

raftmsohani mentioned this pull request Jan 26, 2023

Nginx - Increase security and reliability #2370

Closed

28 tasks

andrew-jameson changed the base branch from develop to 1544-Nginx-with-domain April 10, 2023 19:41

andrew-jameson closed this Apr 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/1544 add nginx configuration increase security performance #2173

Feat/1544 add nginx configuration increase security performance #2173

raftmsohani commented Oct 3, 2022 •

edited

Loading

codecov bot commented Nov 27, 2022 •

edited

Loading

andrew-jameson Jan 4, 2023

andrew-jameson commented Jan 6, 2023

raftmsohani Jan 9, 2023

andrew-jameson Jan 9, 2023 •

edited

Loading

raftmsohani Jan 10, 2023

andrew-jameson Jan 10, 2023

raftmsohani Jan 26, 2023

andrew-jameson Jan 9, 2023

andrew-jameson Jan 9, 2023

raftmsohani Jan 26, 2023

andrew-jameson Jan 9, 2023

raftmsohani Jan 26, 2023

raftmsohani Jan 26, 2023

andrew-jameson Jan 9, 2023

raftmsohani Jan 9, 2023

raftmsohani Jan 26, 2023

andrew-jameson Jan 9, 2023

raftmsohani Jan 9, 2023

raftmsohani Jan 26, 2023

andrew-jameson Jan 9, 2023

raftmsohani Jan 26, 2023

raftmsohani commented Jan 9, 2023

raftmsohani Jan 9, 2023

raftmsohani Jan 26, 2023

raftmsohani Jan 10, 2023

andrew-jameson commented Jan 10, 2023

raftmsohani commented Jan 10, 2023 •

edited

Loading

raftmsohani Jan 10, 2023

andrew-jameson commented Apr 10, 2023


		In general for TDRS, the Nginx handles incoming HTTP requests to both frontend and backend. The frontend requests are handled directly while the backend requests are forwarded to the backend server.

		![Cloud.gov Architecture](./src/arch1.jpg)

	set $CSP "${CSP}script-src-elem 'self' http://localhost:* http://www.w3.org;";
	set $CSP "${CSP}script-src-elem 'self' http://localhost:*;";

		@@ -117,6 +117,7 @@ update_backend()
		cf map-route tdp-backend-prod api-tanfdata.acf.hhs.gov

Feat/1544 add nginx configuration increase security performance #2173

Feat/1544 add nginx configuration increase security performance #2173

Conversation

raftmsohani commented Oct 3, 2022 • edited Loading

Summary of Changes

How to Test

Deliverables

codecov bot commented Nov 27, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

andrew-jameson commented Jan 6, 2023

Choose a reason for hiding this comment

andrew-jameson Jan 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raftmsohani commented Jan 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrew-jameson commented Jan 10, 2023

raftmsohani commented Jan 10, 2023 • edited Loading

Choose a reason for hiding this comment

andrew-jameson commented Apr 10, 2023

raftmsohani commented Oct 3, 2022 •

edited

Loading

codecov bot commented Nov 27, 2022 •

edited

Loading

andrew-jameson Jan 9, 2023 •

edited

Loading

raftmsohani commented Jan 10, 2023 •

edited

Loading