CodeQL Agent for Docker

CodeQL Agent is a project aimed at automating the use of CodeQL. The project helps create database and execute CodeQL analysis. CodeQL Agent is a Docker image.

CodeQL Agent for Docker is also the base image of CodeQL Agent for Visual Studio Code - an extension for Visual Studio Code that simplifies CodeQL usage and executes code scanning automatically.

The CodeQL Agent image is released on Docker Hub under the name doublevkay/codeql-agent. You can use it without building locally.

Contents:

• What is this for?
• Getting started
• Examples usage
• Supported options
• Build
• How does it work?
• Support
• Contributing
• Contributors
• Release notes
• License

What is this for?

CodeQL Agent for Docker provides these key features:

• Detecting language automatically.
• Creating CodeQL database.
• Executing CodeQL database analysis.
• Auto sync the latest version of CodeQL CLI and CodeQL library.

Getting Started

Bind mounts the source, the results folder and run codeql-agent image with the following docker command.

docker run --rm --name codeql-agent-docker \
  -v "$PWD:/opt/src" \
  -v "$PWD/codeql-agent-results:/opt/results" \
  doublevkay/codeql-agent

You also can specify more options to run CodeQL Agent. See Supported options for more details.

Supported options

You can set environment variables to use the following supported options:

Variable	Description
LANGUAGE	Value <language>. Set project language to build database or execute SAST. The <language> must be: go, java, cpp, csharp, python, javascript, ruby.
USERID	Value <id>. Set the owner of the results folder to <id>.
GROUPID	Value <group_id>. Set the group owner of the results folder to <group_id>.
THREADS	Value <number_of_threads>. Use this many threads to build database and evaluate queries. Defaults to 1. You can pass 0 to use one thread per core on the machine.
OVERWRITE_FLAG	Value --overwrite. Enable/disable overwrite database when database path exists and not an empty directory. This flag is useful for forcibly rebuilding the database.
QS	Value <queries-suite>. Specify a list of queries to run over your database. The default value is <language>-security-extended.qls. For more details, please see Analyzing databases with the CodeQL CLI.
SAVE_CACHE_FLAG	Value --save-cache. Aggressively save intermediate results to the disk cache. This may speed up subsequent queries if they are similar. Be aware that using this option will greatly increase disk usage and initial evaluation time.
ACTION	Value create-database-only. Creating CodeQL database only without executing CodeQL analysis.
COMMAND	Value <command>. The variable used when you create a CodeQL database for one or more compiled languages, omit if the only languages requested are Python and JavaScript. This specifies the build commands needed to invoke the compiler. If you don't set this variable, CodeQL will attempt to detect the build system automatically, using a built-in autobuilder.

---

Disclaimer: CodeQL Agent directly forwards these options to the command arguments while running the container. Please take it as your security responsibility.

Examples usage

Basic code scanning.

docker run --rm --name codeql-agent-docker \
  -v "$PWD:/opt/src" \
  -v "$PWD/codeql-agent-results:/opt/results" \
  doublevkay/codeql-agent

Code scanning with maximum threads available.

docker run --rm --name codeql-agent-docker \
  -v "$PWD:/opt/src" \
  -v "$PWD/codeql-agent-results:/opt/results" \
  -e "THREADS=0" \
  doublevkay/codeql-agent

Create database only.

docker run --rm --name codeql-agent-docker \
  -v "$PWD:/opt/src" \
  -v "$PWD/codeql-agent-results:/opt/results" \
  -e "ACTION=create-database-only" \
  doublevkay/codeql-agent

Specify the queries suite for Java source.

docker run --rm --name codeql-agent-docker \
  -v "$PWD:/opt/src" \
  -v "$PWD/codeql-agent-results:/opt/results" \
  -e "LANGUAGE=java" \
  -e "QS=java-security-and-quality.qls" \
  doublevkay/codeql-agent

Change owner of the results folder.

Because CodeQL Agent runs the script as root in Docker containers. So maybe you need to change the results folder owner to your own.

docker run --rm --name codeql-agent-docker \
  -v "$PWD:/opt/src" \
  -v "$PWD/codeql-agent-results:/opt/results" \
  -e "USERID=$(id -u ${USER})" -e "GROUPID=$(id -g ${USER}) \
  doublevkay/codeql-agent

Specify the Java version and the build database command

By default, we use JDK 11 and Maven 3.6.3 for the CodeQL agent image. We can change the versions of Java and Maven by mounting a volume and setting the JAVA_HOME and MAVEN_HOME environment variables in the CodeQL agent container. For example:

1. Create a Dockerfile (named Dockerfile-java) for the specific versions of Java and Maven, and place it in the directory that will be used for mounting later:

    FROM --platform=amd64 maven:3-jdk-8-slim
   
    RUN mkdir -p /opt/jdk/ /opt/maven/
   
    RUN cp -r $JAVA_HOME/* /opt/jdk/
   
    RUN cp -r $MAVEN_HOME/* /opt/maven/
   
    CMD ["echo"]

2. Build and run the Docker container, mounting the JDK and Maven directories to the respective volumes:

    docker buildx build -t codeql-java -f Dockerfile-java .
    docker run --rm  -v "jdkvol:/opt/jdk" -v "mavenvol:/opt/maven" codeql-java

3. Finally, run codeql-agent container with mounted volumes and set env variable JAVA_HOME, MAVEN_HOME to the mounted volumes

docker run --rm --name codeql-agent-docker \
  -v "$PWD:/opt/src" \
  -v "$PWD/codeql-agent-results:/opt/results" \
  -v "jdkvol:/opt/jdk" \
  -v "mavenvol:/opt/maven" \
  -e "LANGUAGE=java" \
  -e "JAVA_HOME=/opt/jdk" \
  -e "MAVEN_HOME=/opt/maven" \
  -e "COMMAND=mvn clean install" \
  doublevkay/codeql-agent

Build

You can use CodeQL Agent Image on Docker Hub or customize and build it locally.

# Build codeql-agent docker image locally 
cd codeql-agent
docker build -t codeql-agent .

How does it work?

CodeQL Agent is a Docker image. The following steps are done to achieve the goals of automating the use of CodeQL.

Setting up environment

In this step, the image prepares the environment for executing CodeQL. It includes: using Ubuntu base image; downloading and installing CodeQL Bundle (which contains the CodeQL CLI and the precompiled library queries to reduce the CodeQL execution time); installing necessary softwares such as java, maven, nodejs, typescript,... to create a CodeQL database successfully.

Detecting language

CodeQL Agent uses github/linguist to detect the source code language.

Creating database

CodeQL Agent runs the CodeQL create database command.

codeql database create --threads=$THREADS --language=$LANGUAGE $COMMAND $DB -s $SRC $OVERWRITE_FLAG

Specifying query suites

Analyzing databases requires specifying a query suite. According to the goals of application static application security testing (SAST) goals, CodeQL Agent uses <language>-security-extended.qls as the default query suite.

Analyzing database

CodeQL Agent runs the CodeQL database analysis command.

codeql database analyze --format=$FORMAT --threads=$THREADS $SAVE_CACHE_FLAG --output=$OUTPUT/issues.$FORMAT $DB $QS

Converting result format

CodeQL Agent will convert the CodeQL result from SARIF format to Security Report Schemas (provided by Gitlab). This step is done by mapping the fields of two formats. The details of implementation are in the sarif2sast script. You can use this script independently as a workaround to solve the Gitlab Issue 118496.

Credits

This repo is based on microsoft/codeql-container and j3ssie/codeql-docker with more function options. Specifically:

• Enhance environment setup to increase reliability.
• Automatically detect language.
• Support helpful CodeQL options.
• Support Java language.

Support

You can open an issue on the GitHub repo

Contributing

Contributions are always welcome! Just create a pull request or contact me

Contributors

Release Notes

See details

License

CodeQL Agent is use CodeQL CLI as the core engine. Please follow the GitHub CodeQL Terms and Conditions and take it as your own responsibility.