Breaking changes for impersonated_credentials between 1.6.3 and 1.7.0+

[jonas-p]

Environment details

• OS: MacOS Mojave
• Python version: 3.7.3
• pip version: 19.0.3
• google-auth version: 1.6.3, 1.7.0, 1.10.0

The default credentials are my user credentials (with the ServiceAccountTokenCreator role on the service account).

Steps to reproduce

The following code works and produces a valid access token for the service account in version 1.6.3.

from google.auth import impersonated_credentials, default
from google.auth.transport.requests import Request

sa = "<sa>@<project>.iam.gserviceaccount.com"
source_credentials, _ = default()
scopes = ["https://www.googleapis.com/auth/cloud-platform"]
creds = impersonated_credentials.Credentials(
    source_credentials=source_credentials, target_principal=sa, target_scopes=scopes
)

creds.refresh(Request())
print(creds.token)

After upgrading to 1.10.0, it fails to authenticate the service account due to invalid scopes.

Traceback (most recent call last):
  File "main.py", line 13, in <module>
    creds.refresh(Request())
  File "/google/lib/python3.7/site-packages/google/auth/impersonated_credentials.py", line 218, in refresh
    self._update_token(request)
  File "/google/lib/python3.7/site-packages/google/auth/impersonated_credentials.py", line 234, in _update_token
    self._source_credentials.refresh(request)
  File "/google/lib/python3.7/site-packages/google/oauth2/credentials.py", line 152, in refresh
    self._scopes,
  File "/google/lib/python3.7/site-packages/google/oauth2/_client.py", line 241, in refresh_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/google/lib/python3.7/site-packages/google/oauth2/_client.py", line 115, in _token_endpoint_request
    _handle_error_response(response_body)
  File "/google/lib/python3.7/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response
    raise exceptions.RefreshError(error_details, response_body)
google.auth.exceptions.RefreshError: ('invalid_scope: Bad Request', '{\n  "error": "invalid_scope",\n  "error_description": "Bad Request"\n}')

I traced this back to an google-auth upgrade from 1.6.3 to 1.7.0 (same error occurs).

[gpoulin]

Facing the same issue. Seems to be related with the fact that impersonated_credentials set the source credentials scopes to ['https://www.googleapis.com/auth/iam'] which is undocumented ref.

A work around is to reset the scope of the source credential

user_cred, _ = default()
credentials = Credentials(user_cred, sa, scopes)
credentials._source_credentials._scopes = user_cred.scopes
creds.refresh(Request())

[tylervick]

Thanks @gpoulin - I confirmed the workaround as well.

Seeing the exact same issue - macOS impersonated creds scopes break on google-auth>1.6.3

[patriknordlen]

Adding some more context to this, the scope was always there but scopes were not actually requested until this change was made:
49a18c4?diff=split#diff-1b020acb6247d690d2ca64fcb04c8f53R241-R242

[busunkim96]

Thank you all for the investigation and details you've provided.

The general documentation for impersonated credentials lives here.

These docs state that auth/iam/ scope and the auth/cloud-platform/ scopes are valid. It doesn't go into detail as to if there's a difference between the two.

scope: The OAuth 2.0 scope for the request. The following scopes are valid when calling the generateAccessToken API:
* https://www.googleapis.com/auth/iam
* https://www.googleapis.com/auth/cloud-platform

As @gpoulin points out the scope is not mentioned on the list of OAuth2 scopes at https://developers.google.com/identity/protocols/googlescopes, but it may just be out of date.

I will do some more investigation internally and update this issue.

[busunkim96]

Alright, so I think I've figured out the root cause. Impersonated credentials modify the scope of the source credential. This follows the guide here.

google-auth-library-python/google/auth/impersonated_credentials.py

Line 208 in 31299e2

self._source_credentials._scopes = _IAM_SCOPE

Howecer google.oauth2.credentials cannot have their scopes modified after creation, so the source credential refresh fails with invalid_scope.

google-auth-library-python/google/auth/impersonated_credentials.py

Lines 224 to 252 in 31299e2

	def _update_token(self, request):
	"""Updates credentials with a new access_token representing
	the impersonated account.
	Args:
	request (google.auth.transport.requests.Request): Request object
	to use for refreshing credentials.
	"""
	# Refresh our source credentials.
	self._source_credentials.refresh(request)
	body = {
	"delegates": self._delegates,
	"scope": self._target_scopes,
	"lifetime": str(self._lifetime) + "s",
	}
	headers = {"Content-Type": "application/json"}
	# Apply the source credentials authentication info.
	self._source_credentials.apply(headers)
	self.token, self.expiry = _make_iam_token_request(
	request=request,
	principal=self._target_principal,
	headers=headers,
	body=body,
	)

google-auth-library-python/google/oauth2/service_account.py

Lines 232 to 239 in 31299e2

	@property
	def requires_scopes(self):
	"""Checks if the credentials requires scopes.
	Returns:
	bool: True if there are no scopes set otherwise False.
	"""
	return True if not self._scopes else False