cicd

CI/CD for CSA on Google Security Operations

Overview

The python folder contains Python helper scripts to gather rules from a local folder and from the remote Google Security Operations instance, and to compare them, uploading changed or new files to the Google Security Operations instance.

Pre-installation

Pre-requisites can be installed by copying the files from the folder locally, and running pip3 install -r requirements.txt

To run this script you will need a Google SecOps API Service Account JSON file. This can be obtained from your Google Security Operations account team.

Usage

First export the path to your Google SecOps API key like this:

export PATH_TO_KEY=~/malachite-abc-7ba40dd4f123.json

Now run the script like this:

python3 rules.py -c $PATH_TO_KEY -l ../../rules -m

The command line arguments are described below:

usage: rules.py [-h] -c CREDENTIALS_FILE -l LOCAL_PATH [-m]

optional arguments:
  -h, --help            show this help message and exit
  -c CREDENTIALS_FILE, --credentials_file CREDENTIALS_FILE
                        path to credentials file
  -i CREDENTIALS_INFO, --credentials_info CREDENTIALS_INFO
                        service account credentials info (alternative to file)
  -e CREDENTIALS_ENV, --credentials_env CREDENTIALS_ENV
                        service account credentials info stored in environment
                        variable
  -l LOCAL_PATH, --local_path LOCAL_PATH
                        local rules path
  -m, --make_changes    fix any differences, if omitted then just report
                        differences
  -s, --silent          supress error messages
  -r REGION, --region REGION
                        Google Security Operations instance region (leave blank for US)

NOTE: if you provide more than one of CREDENTIALS_FILE, CREDENTIALS_INFO, CREDENTIALS_ENV, then the order of precedence will be file -> info -> environment variable.

Example output from reporting (omitting -m flag):

{
  "matched_rules": 6,
  "rules_to_update": 1,
  "non_existent_rules": 2,
  "remote_rules_total": 76,
  "local_rules_total": 9
}

Example output from making changes (including -m flag):

{
  "rules_uploaded": 1,
  "rules_added": 2
}

Using in a GitHub Actions pipeline

The chronicle-rules-cicd.yml file located in the github-actions folder in this repository contains an example of using this Python script to push updates or new detection content to a Google Security Operations instance. There are two variables to edit in this script:

Variable Name	Description	Example Value
region	The region for your Google Security Operations instance	us
rules_path	The relative path from the root of the repository containing the YARA-L rules to work with	rules/yaral

These should be modified for your environment, and the file should be placed in a .github/workflows folder in the root of your repository.

In addition to this, the pipeline file requires a secret to be created on your GitHub repository. Details for creating this can be found here. The secret in this case should be named SA_CREDENTIAL. The value is derived from the contents of your Google SecOps API key, but line breaks should be removed from the file, and the " character should also be replaced with \". The resulting string can then be pasted into the Secrets UI in GitHub.

Now whenever a change is written to the repository the contents of the passed rules folder will be checked and updated/uploaded on the Google Security Operations instance. To learn more GitHub Actions workflows, see here.

Using in a Google Cloud Build pipeline

The cloudbuild.yaml file located in the cloudbuild folder in this repository contains an example of using this Python script to push updates or new detection content to a Google Security Operations instance using Google Cloud Build.

The pipeline file requires a secret to be created in Secrets Manager, and for this to be made available to the service account running the build pipeline. Details for creating this can be found here. The value can be copy/pasted from the contents of your Google SecOps API key into the Secrets Manager UI or API.

Your repository should be added in Cloud Build, and a trigger created following this document, with the below substitutions added.

There are four user-defined substitutions to create to support this script:

Substitution Name	Description	Example Value
_REGION	The region for your Google Security Operations instance	us
_RULES_PATH	The relative path from the root of the repository containing the YARA-L rules to work with	rules/yaral
_PROJECT_ID	The project ID containing the secret created earlier	my-project-id
_SECRET_NAME	The name of the secret created earlier	bk_api_credential

More detail on substitutions can be found here.

The cloudbuild.yaml file should be placed in the root of your repository.

Using in a Azure DevOps pipeline

The azure-pipelines.yml file located in the azure-devops folder in this repository contains an example of using this Python script to push updates or new detection content to a Google Security Operations instance using Azure DevOps Pipelines.

There are two variables to edit in this script:

Variable Name	Description	Example Value
region	The region for your Google Security Operations instance	us
rules_path	The relative path from the root of the repository containing the YARA-L rules to work with	rules/yaral

These should be modified for your environment, and the file should be placed in the root of your repository.

The pool name value should also be updated in the pipelines file, to match the agent pool you want to use to run the code.

In addition to this, the pipeline file requires a secret to be created on your Azure DevOps project. Details for creating this can be found here. The secret in this case should be named SA_CREDENTIAL. The value is the contents of your Google SecOps API key, which can just be copy/pasted into the UI, shown in the linked document above, from the JSON file containing the key.

Now whenever a change is written to the repository the contents of the passed rules folder will be checked and updated/uploaded on the Google Security Operations instance.