Build local copy of Digital Analytics Program #11097

aduth · 2024-08-16T15:00:38Z

🛠 Summary of changes

Updates usage of Digital Analytics Program to build and serve a local copy of the script.

Why?

Serving from local copy reduces security risks inherent to loading scripts from third-party CDNs, even if the third-party is a trusted government source
- Allows further strictening of content security policy
Incorporating into existing build toolchain adds support for:
- Subresource integrity (see related Slack thread)
- Optimized minification from bundler dead code elimination (see upstream Use bundler to build minified JavaScript digital-analytics-program/gov-wide-code#118)
Adds ability to fast-track patches to the code (see example upstream Load Google Tag Manager script as asynchronous digital-analytics-program/gov-wide-code#121 )
Sets precedent for copying external assets for local use, hopefully

📜 Testing Plan

Add to config/application.yml:

participate_in_dap: true

Verify analytics script loads successfully:

Go to http://localhost:3000
Verify on errors in local console, and that Google Analytics loads successfully. You should see some warnings about Third-party cookies being blocked in future versions of Chrome

aduth · 2024-08-16T15:01:08Z

app/javascript/packages/analytics/digital-analytics-program.patch

+785d785
+< var piiRegex = [];
+900c900
+<         piiRegex.forEach(function (pii) {
+---
+>         window.piiRegex.forEach(function (pii) {


See: digital-analytics-program/gov-wide-code#118 (comment)

app/helpers/script_helper.rb

app/services/uri_service.rb

app/javascript/packages/analytics/.gitignore

app/javascript/packages/analytics/package.json

app/helpers/script_helper.rb

See: #11097 (comment) Co-authored-by: Zach Margolis <zachmargolis@users.noreply.github.com>

aduth · 2024-08-16T19:14:32Z

Optimized minification from bundler dead code elimination (see upstream Use bundler to build minified JavaScript digital-analytics-program/gov-wide-code#118)

We can also benefit from our Brotli compression, compared to Gzip used with the hosted version.

Before:

curl --silent "https://dap.digitalgov.gov/Universal-Federated-Analytics-Min.js?agency=GSA&subagency=TTS" | gzip-size

8.24 kB

After:

NODE_ENV=production yarn build && brotli-size public/packs/digital-analytics-program*.js

6.78 kB

Diff: -1.46kb (-17.6%)

app/javascript/packages/analytics/Makefile

changelog: Internal, Performance, Optimize loading of Digital Analytics Program script

DAP script has too many side effects and manipulates DOM in a way that's hard to clean up. Absent some other way to create an isolated DOM, at least check that the script can be evaluated without syntax errors.

See: #11097 (comment) Co-authored-by: Zach Margolis <zachmargolis@users.noreply.github.com>

See: https://github.com/18F/identity-idp/blob/main/docs/frontend.md#naming-conventions

Co-authored-by: Zach Margolis <zachmargolis@users.noreply.github.com>

In production environments, `yarn install --production` won't make workspaces available, so trying to use `yarn workspace ...` will fail. This achieves the same effect without relying on the workspace being "installed"

In CI Dockerfile, app files aren't available at time of install. This also aligns closer to how browsers.json is generated

Defer to sub-process to decide what needs to be done

Avoid dependency on package code existing during install step

aduth commented Aug 16, 2024

View reviewed changes

app/helpers/script_helper.rb Outdated Show resolved Hide resolved

zachmargolis reviewed Aug 16, 2024

View reviewed changes

app/services/uri_service.rb Outdated Show resolved Hide resolved

zachmargolis reviewed Aug 16, 2024

View reviewed changes

app/javascript/packages/analytics/.gitignore Outdated Show resolved Hide resolved

app/javascript/packages/analytics/package.json Show resolved Hide resolved

aduth force-pushed the aduth-dap-local branch from dc3e8da to 4c4fb99 Compare August 16, 2024 18:48

zachmargolis reviewed Aug 16, 2024

View reviewed changes

app/helpers/script_helper.rb Outdated Show resolved Hide resolved

aduth added a commit that referenced this pull request Aug 16, 2024

Treat url_params as distinct property

c05c3e0

See: #11097 (comment) Co-authored-by: Zach Margolis <zachmargolis@users.noreply.github.com>

aduth marked this pull request as ready for review August 16, 2024 19:09

aduth requested a review from mitchellhenke August 16, 2024 19:09

zachmargolis approved these changes Aug 16, 2024

View reviewed changes

aduth commented Aug 16, 2024

View reviewed changes

app/javascript/packages/analytics/Makefile Outdated Show resolved Hide resolved

aduth marked this pull request as draft August 19, 2024 14:33

aduth and others added 17 commits August 20, 2024 14:49

Build local copy of Digital Analytics Program

2df33e9

changelog: Internal, Performance, Optimize loading of Digital Analytics Program script

Remove CSP exceptions for DAP CDN

249fa5a

Download and patch DAP as postinstall

8cf8e8f

Consistently return string value from UriService#add_params

fb1e070

Add spec to load DAP script

49948fa

Update DAP script test to check syntax error on load

cf3aa92

DAP script has too many side effects and manipulates DOM in a way that's hard to clean up. Absent some other way to create an isolated DOM, at least check that the script can be evaluated without syntax errors.

Add DAP script to ESLint ignore

b921c19

Add test for url_params behavior

028d6b0

Better coverage for combination url_params and attributes

329cbed

Treat url_params as distinct property

5071007

See: #11097 (comment) Co-authored-by: Zach Margolis <zachmargolis@users.noreply.github.com>

Use conventional capitalization

272f969

See: https://github.com/18F/identity-idp/blob/main/docs/frontend.md#naming-conventions

Separate download and patch tasks to cache-bust by SHA

38f68ba

Ignore DAP script in TypeScript checks

3f95f33

Use more Makefile wizardry to generate compiled DAP

843fdcd

Co-authored-by: Zach Margolis <zachmargolis@users.noreply.github.com>

Run analytics postinstall using Yarn cwd

ce88019

In production environments, `yarn install --production` won't make workspaces available, so trying to use `yarn workspace ...` will fail. This achieves the same effect without relying on the workspace being "installed"

Try avoiding default NPM lifecycle script names

bb8eb54

Test using full install in cwd package

a9afae0

aduth added 7 commits August 20, 2024 14:49

Try incorporating analytics tasks in build:js

af338d6

In CI Dockerfile, app files aren't available at time of install. This also aligns closer to how browsers.json is generated

Combine make tasks in build:js

7ca7d8a

Force analytics task in top-level Makefile

f9621b8

Defer to sub-process to decide what needs to be done

Split install and build tasks for analytics

4271a1f

Avoid dependency on package code existing during install step

Copy Makefile earlier in Docker images

453ec58

Try complete build if folder available

6a71e47

Revert to original postinstall approach

c626bf6

aduth force-pushed the aduth-dap-local branch from ae80d52 to c626bf6 Compare August 20, 2024 18:55

aduth marked this pull request as ready for review August 20, 2024 19:27

aduth merged commit 85df0cf into main Aug 20, 2024
2 checks passed

aduth deleted the aduth-dap-local branch August 20, 2024 20:27

aduth mentioned this pull request Aug 21, 2024

Use defer when loading GTM script via patched DAP #11126

Merged

solipet mentioned this pull request Aug 22, 2024

Deploy RC 408 to Production #11136

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build local copy of Digital Analytics Program #11097

Build local copy of Digital Analytics Program #11097

aduth commented Aug 16, 2024

aduth Aug 16, 2024

aduth commented Aug 16, 2024

Build local copy of Digital Analytics Program #11097

Build local copy of Digital Analytics Program #11097

Conversation

aduth commented Aug 16, 2024

🛠 Summary of changes

📜 Testing Plan

aduth Aug 16, 2024

Choose a reason for hiding this comment

aduth commented Aug 16, 2024