- Project State-: Maintained
For more information on project states and SLAs, see this documentation.
This InSpec resource pack uses the native Google Cloud Platform (GCP) support in InSpec and provides the required resources to write tests for GCP.
This implementation was inspired on the ideas by Martez Reed.
Download the SDK and run the installation:
./google-cloud-sdk/install.sh
gcloud auth application-default loginIf successful, this should be similar to:
$ cat ~/.config/gcloud/application_default_credentials.json
{
"client_id": "764086051850-6qr4p6gpi6hn50asdr.apps.googleusercontent.com",
"client_secret": "d-fasdfasdfasdfaweroi23jknrmfs;f8sh",
"refresh_token": "1/asdfjlklwna;ldkna'dfmk-lCkju3-yQmjr20xVZonrfkE48L",
"type": "authorized_user"
}While InSpec can use user accounts for authentication, Google Cloud documentation recommends using service accounts.
- Go to
Consolein the Google Cloud. - Click on
Service Accounts. - Click on
Create Service Accountsand give the required details. - Click on
Done. - Now Click on
KeysTab. - Click on
Add Key. - Click on
Create New Key. - Select the Key type as
JSON. - Click on
Create. - The Service Account Key will be downloaded. For example:
myproject-1-feb7993e8660.jsonmyproject-1-feb7993e8660.json - Move the key to the
~/.config/gcloudfolder. If the file gets downloaded in theDownloadsfolder, use the below command.
mv ~/Downloads/myproject-1-feb7993e8660.json ~/.config/gcloud/- The json credential file for a service account looks like this:
$ cat ~/.config/gcloud/myproject-1-feb7993e8660.json
{
"type": "service_account",
"project_id": "myproject-1",
"private_key_id": "eb45b2fc0c33ea9b6fa212aaa08b1ed814bf8660",
"private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvwIBADAN3662...fke9n6LAf268E/4EWhIzg==\n-----END PRIVATE KEY-----\n",
"client_email": "auto-testing@myproject-1.iam.gserviceaccount.com",
"client_id": "112144174133171863632",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/auto-testing%40myproject-1.iam.gserviceaccount.com"
}- Now Setup the environmental variable for the 'GOOGLE_APPLICATION_CREDENTIALS'. And InSpec can be instructed to use it by setting this ENV variable prior to running
inspec exec:
$ export GOOGLE_APPLICATION_CREDENTIALS='~/.config/gcloud/myproject-1-feb7993e8660.json'If still error comes, please check your IAM roles and permissions.
Since this is an InSpec resource pack, it only defines InSpec resources. It includes example tests only. To easily use the GCP resources in your tests do the following:
$ inspec init profile --platform gcp my-gcp-profile
Create new profile at /Users/spaterson/my-gcp-profile
* Create directory libraries
* Create file README.md
* Create directory controls
* Create file controls/example.rb
* Create file inspec.yml
* Create file attributes.yml
* Create file libraries/.gitkeepgcp_project_id: 'my-gcp-project'The generated inspec.yml file automatically points to the InSpec GCP resource pack:
name: my-gcp-profile
title: My GCP InSpec Profile
version: 0.1.0
inspec_version: '>= 4'
inputs:
- name: gcp_project_id
required: true
description: 'The GCP project identifier.'
depends:
- name: inspec-gcp
url: https://github.com/inspec/inspec-gcp/archive/x.tar.gz
supports:
- platform: gcp(For available inspec-gcp versions, see this list of inspec-gcp versions.)
$ cd my-gcp-profile/
$ inspec exec . -t gcp:// --input-file inputs.yml
Profile: GCP InSpec Profile (my-profile)
Version: 0.1.0
Target: gcp://local-service-account@my-gcp-project.iam.gserviceaccount.com
Target ID: 8123456-12a3-1234-123a-a12s5c5abcx1
âś” gcp-single-region-1.0: Ensure single region has the correct properties.
âś” Region europe-west2 zone_names should include "europe-west2-a"
âś” gcp-regions-loop-1.0: Ensure regions have the correct properties in bulk.
âś” Region asia-east1 is expected to be up
âś” Region asia-east2 is expected to be up
âś” Region asia-northeast1 is expected to be up
âś” Region asia-northeast2 is expected to be up
âś” Region asia-northeast3 is expected to be up
âś” Region asia-south1 is expected to be up
âś” Region asia-south2 is expected to be up
âś” Region asia-southeast1 is expected to be up
âś” Region asia-southeast2 is expected to be up
âś” Region australia-southeast1 is expected to be up
âś” Region australia-southeast2 is expected to be up
âś” Region europe-central2 is expected to be up
âś” Region europe-north1 is expected to be up
âś” Region europe-southwest1 is expected to be up
âś” Region europe-west1 is expected to be up
âś” Region europe-west2 is expected to be up
âś” Region europe-west3 is expected to be up
âś” Region europe-west4 is expected to be up
âś” Region europe-west6 is expected to be up
âś” Region europe-west8 is expected to be up
âś” Region europe-west9 is expected to be up
âś” Region me-west1 is expected to be up
âś” Region northamerica-northeast1 is expected to be up
âś” Region northamerica-northeast2 is expected to be up
âś” Region southamerica-east1 is expected to be up
âś” Region southamerica-west1 is expected to be up
âś” Region us-central1 is expected to be up
âś” Region us-east1 is expected to be up
âś” Region us-east4 is expected to be up
âś” Region us-east5 is expected to be up
âś” Region us-south1 is expected to be up
âś” Region us-west1 is expected to be up
âś” Region us-west2 is expected to be up
âś” Region us-west3 is expected to be up
âś” Region us-west4 is expected to be up
Profile: Google Cloud Platform Resource Pack (inspec-gcp)
Version: 1.10.37
Target: gcp://local-service-account@my-gcp-project.iam.gserviceaccount.com
Target ID: 8123456-12a3-1234-123a-a12s5c5abcx1
No tests executed.
Profile Summary: 2 successful controls, 0 control failures, 0 controls skipped
Test Summary: 36 successful, 0 failures, 0 skippedThe following resources are available in the InSpec GCP Profile
We use several plural resources for this example that loops across all projects and firewall rules. Making use of a plural resource property, we filter firewall rules for direction 'INGRESS' :
title 'Loop over all GCP projects and look at firewalls in INGRESS direction'
control 'gcp-projects-firewalls-loop-1.0' do
impact 1.0
title 'Ensure INGRESS firewalls in all projects have the correct properties using google_compute_firewall for detail.'
google_projects.project_names.each do |project_name|
google_compute_firewalls(project: project_name).where(firewall_direction: 'INGRESS').firewall_names.each do |firewall_name|
describe google_compute_firewall(project: project_name, name: firewall_name) do
its('allowed_ssh?') { should be false }
end
end
end
end
This example assumes there are sufficient privileges to list all GCP projects.
This check ensures that VMs have label must_be_there for each project:
title 'Loop over all GCP projects and ensure all VMs have a particular label'
control 'gcp-projects-zones-vm-label-loop-1.0' do
impact 1.0
title 'Ensure all VMs have must_be_there label key set'
google_projects.project_names.each do |project_name|
google_compute_zones(project: project_name).zone_names.each do |zone_name|
google_compute_instances(project: project_name, zone: zone_name).instance_names.each do |instance_name|
describe google_compute_instance(project: project_name, zone: zone_name, name: instance_name) do
its('labels_keys') { should include 'must_be_there' }
end
end
end
end
end
This example verifies there are sufficient privileges to list all regions.
next unless google_compute_regions(project: gcp_project_id).resource_failed?
google_compute_regions(project: gcp_project_id).region_names.each do |region_name|
describe google_compute_region(project: gcp_project_id, region: region_name) do
it { should be_up }
end
end
if google_compute_regions(project: gcp_project_id).resource_failed?
puts google_compute_regions(project: gcp_project_id).resource_exception_message
puts google_compute_regions(project: gcp_project_id,name: region_name).pretty_inspect
end
This example assumes there are sufficient privileges to list all GCP projects.
- Create a new GCP project
- Ensure this is currently set following: https://cloud.google.com/shell/docs/examples
$ gcloud config set project <project-name>
$ gcloud config list project- Enable billing for your new project
- Enable various services necessary to run the tests:
$ gcloud services enable compute.googleapis.com \
sourcerepo.googleapis.com \
dns.googleapis.com \
container.googleapis.com \
cloudkms.googleapis.com \
cloudbuild.googleapis.com- Environment variables can be used to specify project details e.g.
export GCP_PROJECT_NAME=<project-name>
export GCP_PROJECT_NUMBER=<project-number>
export GCP_PROJECT_ID=<project-id>Some resources require elevated privileges to create in GCP. These are disabled by default but can be activated via:
export GCP_ENABLE_PRIVILEGED_RESOURCES=1This takes effect during the "plan" task as described in the next section. Affected terraform resources are included/excluded and associated inspec tests enabled/disabled accordingly.
The resource pack defaults to region europe-west2. If a different region is desired e.g. us-central1, the following variables should be set:
export GCP_LOCATION="us-central1"
export GCP_ZONE="us-central1-a"
export GCP_LB_REGION="us-central1"
export GCP_LB_ZONE="us-central1-a"
export GCP_LB_ZONE_MIG2="us-central1-b"
export GCP_LB_ZONE_MIG3="us-central1-c"
export GCP_KUBE_CLUSTER_ZONE="us-central1-a"
export GCP_KUBE_CLUSTER_ZONE_EXTRA1="us-central1-b"
export GCP_KUBE_CLUSTER_ZONE_EXTRA2="us-central1-c"
export GCP_CLOUD_FUNCTION_REGION="us-central1"Other regions can be targeted by updating the above. For example, see https://cloud.google.com/compute/docs/regions-zones/ for suitable values.
-
Ensure the
In-use IP addressesquota is set to 20 or above for chosen region -
Run the integration tests via:
bundle install && bundle exec rake test:integrationAlternatively, finer grained rake tasks are also available. Executing these in order is the same as the above command:
- Initialize local workspace (terraform init)
bundle exec rake test:init_workspace- Plan integration tests - ensures variables are set for Inspec and Terraform, runs "terraform plan"
bundle exec rake test:plan_integration_tests- Set up integration tests - actually creates the resources in GCP (terraform apply)
bundle exec rake test:setup_integration_tests- Run integration tests - runs the tests (inspec exec)
bundle exec rake test:run_integration_tests- Clean up integration tests - removes GCP resources (terraform destroy)
bundle exec rake test:cleanup_integration_testsIf an error such as the below occurs when running "inspec exec" on a newly created GCP profile:
libraries/google_compute_instance.rb:26:in `block in initialize': undefined method `gcp_compute_client' for #<Train::Transports::Local::Connection:0x00007fcasdf1a532d0> (NoMethodError)
Check that the GCP transport is being specified as below:
$ inspec exec . -t gcp://
This tells the underlying transport layer (train) to use GCP.
InSpec relies on the GCP API's to verify the settings. Therefore, it requires access to the API. If you try to access an API via an InSpec resource that is not enabled in your account, then you see an error like:
googleapi: Error 403: Access Not Configured. Compute Engine API has not been used in project 41111111111 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview then retry.
The terraform templates generate sufficient resources to require an increase to default in_use IP addresses. Normally new projects have 10, increasing this to 20 or higher should be sufficient.
To find this setting, log in to the GCP web interface and go to IAM and admin->Quotas and look for "Compute Engine API In-use IP addresses". From here you can "Edit quotas" to request more.
Changed Quota:
+----------------------+------------------+
| Region: europe-west2 | IN_USE_ADDRESSES |
+----------------------+------------------+
| Changes | 8 -> 64 |
+----------------------+------------------+
Sometimes there can be occasional errors when performing the cleanup rake task. This happens when resources are already deleted and can be ignored.
A guide on upgrading to version 1.0 can be found here
The InSpec GCP resources are community supported. For bugs and features, please open a github issue and label it appropriately.
This implementation is inspired by inspec-azure and inspec-gcp