From 2142db314666f298071ae30a7419b00d48d87476 Mon Sep 17 00:00:00 2001 From: Leo <39062083+lsirac@users.noreply.github.com> Date: Thu, 18 Feb 2021 11:35:09 -0800 Subject: [PATCH] docs: add instructions for using workload identity federation (#564) * docs: add instructions for using workload identity federation * docs: address review comments --- README.md | 219 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 219 insertions(+) diff --git a/README.md b/README.md index 9d11eef77..7ef7021a4 100644 --- a/README.md +++ b/README.md @@ -106,6 +106,12 @@ access tokens - `ServiceAccountJwtAccessCredentials`: credentials for a Service Account - use JSON Web Token (JWT) directly in the request metadata to provide authorization - `UserCredentials`: credentials for a user identity and consent +- `ExternalAccountCredentials`: base class for credentials using workload identity federation to +access Google Cloud resources from non-Google Cloud platforms +- `IdentityPoolCredentials`: credentials using workload identity federation to access Google Cloud +resources from Microsoft Azure or any identity provider that supports OpenID Connect (OIDC) +- `AwsCredentials`: credentials using workload identity federation to access Google Cloud resources +from Amazon Web Services (AWS) To get Application Default Credentials use `GoogleCredentials.getApplicationDefault()` or `GoogleCredentials.getApplicationDefault(HttpTransportFactory)`. These methods return the @@ -159,6 +165,219 @@ for (Bucket b : storage_service.list().iterateAll()) System.out.println(b); ``` +### Workload Identity Federation + +Using workload identity federation, your application can access Google Cloud resources from +Amazon Web Services (AWS), Microsoft Azure, or any identity provider that supports OpenID Connect +(OIDC). + +Traditionally, applications running outside Google Cloud have used service account keys to access +Google Cloud resources. Using identity federation, your workload can impersonate a service account. +This lets the external workload access Google Cloud resources directly, eliminating the maintenance +and security burden associated with service account keys. + +#### Accessing resources from AWS + +In order to access Google Cloud resources from Amazon Web Services (AWS), the following requirements +are needed: +- A workload identity pool needs to be created. +- AWS needs to be added as an identity provider in the workload identity pool (the Google [organization policy](https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers#restrict) needs to allow federation from AWS). +- Permission to impersonate a service account needs to be granted to the external identity. + +Follow the detailed [instructions](https://cloud.google.com/iam/docs/access-resources-aws) on how to +configure workload identity federation from AWS. + +After configuring the AWS provider to impersonate a service account, a credential configuration file +needs to be generated. Unlike service account credential files, the generated credential +configuration file contains non-sensitive metadata to instruct the library on how to +retrieve external subject tokens and exchange them for service account access tokens. +The configuration file can be generated by using the [gcloud CLI](https://cloud.google.com/sdk/). + +To generate the AWS workload identity configuration, run the following command: + +```bash +# Generate an AWS configuration file. +gcloud iam workload-identity-pools create-cred-config \ + projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$AWS_PROVIDER_ID \ + --service-account $SERVICE_ACCOUNT_EMAIL \ + --aws \ + --output-file /path/to/generated/config.json +``` + +Where the following variables need to be substituted: +- `$PROJECT_NUMBER`: The Google Cloud project number. +- `$POOL_ID`: The workload identity pool ID. +- `$AWS_PROVIDER_ID`: The AWS provider ID. +- `$SERVICE_ACCOUNT_EMAIL`: The email of the service account to impersonate. + +This generates the configuration file in the specified output file. + +You can now [use the Auth library](#using-external-identities) to call Google Cloud +resources from AWS. + +#### Access resources from Microsoft Azure + +In order to access Google Cloud resources from Microsoft Azure, the following requirements are +needed: +- A workload identity pool needs to be created. +- Azure needs to be added as an identity provider in the workload identity pool (the Google [organization policy](https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers#restrict) needs to allow federation from Azure). +- The Azure tenant needs to be configured for identity federation. +- Permission to impersonate a service account needs to be granted to the external identity. + +Follow the detailed [instructions](https://cloud.google.com/iam/docs/access-resources-azure) on how +to configure workload identity federation from Microsoft Azure. + +After configuring the Azure provider to impersonate a service account, a credential configuration +file needs to be generated. Unlike service account credential files, the generated credential +configuration file contains non-sensitive metadata to instruct the library on how to +retrieve external subject tokens and exchange them for service account access tokens. +The configuration file can be generated by using the [gcloud CLI](https://cloud.google.com/sdk/). + +To generate the Azure workload identity configuration, run the following command: + +```bash +# Generate an Azure configuration file. +gcloud iam workload-identity-pools create-cred-config \ + projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$AZURE_PROVIDER_ID \ + --service-account $SERVICE_ACCOUNT_EMAIL \ + --azure \ + --output-file /path/to/generated/config.json +``` + +Where the following variables need to be substituted: +- `$PROJECT_NUMBER`: The Google Cloud project number. +- `$POOL_ID`: The workload identity pool ID. +- `$AZURE_PROVIDER_ID`: The Azure provider ID. +- `$SERVICE_ACCOUNT_EMAIL`: The email of the service account to impersonate. + +This generates the configuration file in the specified output file. + +You can now [use the Auth library](#using-external-identities) to call Google Cloud +resources from Azure. + +#### Accessing resources from an OIDC identity provider + +In order to access Google Cloud resources from an identity provider that supports [OpenID Connect (OIDC)](https://openid.net/connect/), the following requirements are needed: +- A workload identity pool needs to be created. +- An OIDC identity provider needs to be added in the workload identity pool (the Google [organization policy](https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers#restrict) needs to allow federation from the identity provider). +- Permission to impersonate a service account needs to be granted to the external identity. + +Follow the detailed [instructions](https://cloud.google.com/iam/docs/access-resources-oidc) on how +to configure workload identity federation from an OIDC identity provider. + +After configuring the OIDC provider to impersonate a service account, a credential configuration +file needs to be generated. Unlike service account credential files, the generated credential +configuration file contains non-sensitive metadata to instruct the library on how to +retrieve external subject tokens and exchange them for service account access tokens. +The configuration file can be generated by using the [gcloud CLI](https://cloud.google.com/sdk/). + +For OIDC providers, the Auth library can retrieve OIDC tokens either from a local file location +(file-sourced credentials) or from a local server (URL-sourced credentials). + +**File-sourced credentials** +For file-sourced credentials, a background process needs to be continuously refreshing the file +location with a new OIDC token prior to expiration. For tokens with one hour lifetimes, the token +needs to be updated in the file every hour. The token can be stored directly as plain text or in +JSON format. + +To generate a file-sourced OIDC configuration, run the following command: + +```bash +# Generate an OIDC configuration file for file-sourced credentials. +gcloud iam workload-identity-pools create-cred-config \ + projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$OIDC_PROVIDER_ID \ + --service-account $SERVICE_ACCOUNT_EMAIL \ + --credential-source-file $PATH_TO_OIDC_ID_TOKEN \ + # Optional arguments for file types. Default is "text": + # --credential-source-type "json" \ + # Optional argument for the field that contains the OIDC credential. + # This is required for json. + # --credential-source-field-name "id_token" \ + --output-file /path/to/generated/config.json +``` + +Where the following variables need to be substituted: +- `$PROJECT_NUMBER`: The Google Cloud project number. +- `$POOL_ID`: The workload identity pool ID. +- `$OIDC_PROVIDER_ID`: The OIDC provider ID. +- `$SERVICE_ACCOUNT_EMAIL`: The email of the service account to impersonate. +- `$PATH_TO_OIDC_ID_TOKEN`: The file path used to retrieve the OIDC token. + +This generates the configuration file in the specified output file. + +**URL-sourced credentials** +For URL-sourced credentials, a local server needs to host a GET endpoint to return the OIDC token. +The response can be in plain text or JSON. Additional required request headers can also be +specified. + +To generate a URL-sourced OIDC workload identity configuration, run the following command: + +```bash +# Generate an OIDC configuration file for URL-sourced credentials. +gcloud iam workload-identity-pools create-cred-config \ + projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$OIDC_PROVIDER_ID \ + --service-account $SERVICE_ACCOUNT_EMAIL \ + --credential-source-url $URL_TO_GET_OIDC_TOKEN \ + --credential-source-headers $HEADER_KEY=$HEADER_VALUE \ + # Optional arguments for file types. Default is "text": + # --credential-source-type "json" \ + # Optional argument for the field that contains the OIDC credential. + # This is required for json. + # --credential-source-field-name "id_token" \ + --output-file /path/to/generated/config.json +``` + +Where the following variables need to be substituted: +- `$PROJECT_NUMBER`: The Google Cloud project number. +- `$POOL_ID`: The workload identity pool ID. +- `$OIDC_PROVIDER_ID`: The OIDC provider ID. +- `$SERVICE_ACCOUNT_EMAIL`: The email of the service account to impersonate. +- `$URL_TO_GET_OIDC_TOKEN`: The URL of the local server endpoint to call to retrieve the OIDC token. +- `$HEADER_KEY` and `$HEADER_VALUE`: The additional header key/value pairs to pass along the GET +request to `$URL_TO_GET_OIDC_TOKEN`, e.g. `Metadata-Flavor=Google`. + +You can now [use the Auth library](#using-external-identities) to call Google Cloud +resources from an OIDC provider. + +#### Using External Identities + +External identities (AWS, Azure, and OIDC-based providers) can be used with +`Application Default Credentials`. In order to use external identities with Application Default +Credentials, you need to generate the JSON credentials configuration file for your external identity +as described above. Once generated, store the path to this file in the +`GOOGLE_APPLICATION_CREDENTIALS` environment variable. + +```bash +export GOOGLE_APPLICATION_CREDENTIALS=/path/to/config.json +``` + +The library can now choose the right type of client and initialize credentials from the context +provided in the configuration file. + +```java +GoogleCredentials googleCredentials = GoogleCredentials.getApplicationDefault(); + +String projectId = "your-project-id"; +String url = "https://storage.googleapis.com/storage/v1/b?project=" + projectId; + +HttpCredentialsAdapter credentialsAdapter = new HttpCredentialsAdapter(googleCredentials); +HttpRequestFactory requestFactory = new NetHttpTransport().createRequestFactory(credentialsAdapter); +HttpRequest request = requestFactory.buildGetRequest(new GenericUrl(url)); + +JsonObjectParser parser = new JsonObjectParser(GsonFactory.getDefaultInstance()); +request.setParser(parser); + +HttpResponse response = request.execute(); +System.out.println(response.parseAsString()); +``` + +You can also explicitly initialize external account clients using the generated configuration file. + +```java +ExternalAccountCredentials credentials = + ExternalAccountCredentials.fromStream(new FileInputStream("/path/to/credentials.json")); +``` + ## Configuring a Proxy For HTTP clients, a basic proxy can be configured by using `http.proxyHost` and related system properties as documented