test: add lightweight integration tests

.github/workflows/ci.yaml

@@ -20,3 +20,35 @@ jobs:
       - run: yarn install --frozen-lockfile
       - run: yarn build
       - run: yarn test
+
+  e2e:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+        with:
+          node-version: 18
+      # Setup gcloud application default credentials. While the credentials are not actually
+      # used because Google api services are stubbed, instantiating any of the Google node
+      # clients requires the credentials file to exist and be valid.
+      - uses: "google-github-actions/auth@v2"
+        with:
+          credentials_json: "${{ secrets.GCP_CREDENTIALS }}"
+      - uses: google-github-actions/setup-gcloud@v2
+        with:
+          version: ">= 363.0.0"
+      - run: yarn install --frozen-lockfile
+      - run: yarn e2e
+
+  # Join CI jobs into a single status so that only one status needs to be
+  # marked as required under branch protection rules.
+  all_ci_tests:
+    runs-on: ubuntu-latest
+    needs:
+      - test
+      - e2e
+    if: ${{ always() }}
+    steps:
+      - uses: cgrindel/gha_join_jobs@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}

e2e/README.md

@@ -0,0 +1,30 @@
+# e2e tests
+
+Run with
+
+```bash
+yarn e2e
+```
+
+Requires `gcloud` application default credentials. The e2e tests stub google cloud APIs and
+don't actually use the credentials, however the file `~/.config/gcloud/application_default_credentials.json`
+must still exist and be valid for the `@google-cloud` node clients to work properly.
+
+```bash
+gcloud auth application-default login
+```
+
+## Test strategy
+
+The end-to-end tests use a combination of fake and real services.
+
+The full webhook is run locally using [Functions Framework](https://cloud.google.com/functions/docs/functions-framework), which is the same runtime that Google Cloud Functions uses.
+
+Email notifications are sent to an email test service called [Ethereal](https://ethereal.email/), which our email client `nodemailer` natively supports.
+
+GitHub and Google Secrets Manager are stubbed. It may seem counterintuitive to stub GitHub when testing a GitHub app, however, using the real service presents several challenges:
+
+- The Bazel SIG would need to maintain a fleet of other GitHub organizations, repositories, and users for testing, creating additional maintenance burden.
+- Test repositories need to be kept in a pristine state across test runs. Simultaneous test runs could cause conflicts in test results and require that only one individual runs tests at a time.
+
+In the current state, tests can be run locally, simultaneously with other developers with no resource contention. The GitHub API is well-defined and easy enough to stub with test data.