Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azure_rm_keyvaultsecret doesn't reliably allow specified credentials #1009

Closed
markscottwright opened this issue Oct 28, 2022 · 6 comments · Fixed by #1010
Closed

azure_rm_keyvaultsecret doesn't reliably allow specified credentials #1009

markscottwright opened this issue Oct 28, 2022 · 6 comments · Fixed by #1010
Labels
has_pr PR fixes have been made medium_priority Medium priority

Comments

@markscottwright
Copy link
Contributor

markscottwright commented Oct 28, 2022

SUMMARY

If credentials are explicitly specified for azure_rm_keyvaultsecret, but it is possible to retrieve credentials from the environment, the explicitly specified credentials are ignored. In other words, the (default) auth_source of "auto" does not act as documented "When set to auto (the default) the precedence is module parameters -> env -> credential_file -> cli." Instead, the precedence is "env -> credential_file -> cli -> module parameters".

This means that there is no way to be certain that specified module parameters will be used.

The error is in line 207:

elif self.module.params['auth_source'] in ['auto', 'cli']:

The code on that line should probably be something like:
"if self.module.params['auth_source'] == 'cli' or (self.module.params['auth_source'] == 'auto' and self.credentials['client_id'] is None and self.credentials['secret'] is None):"

ISSUE TYPE
  • Bug Report
COMPONENT NAME

azure_rm_keyvaultsecret

ANSIBLE VERSION
ansible [core 2.13.5]
  config file = /home/mark/PRIVATE/ansible.cfg
  configured module search path = ['/home/mark/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mark/PRIVATE/venv/lib/python3.10/site-packages/ansible
  ansible collection location = /home/mark/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/mark/PRIVATE/venv/bin/ansible
  python version = 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
  jinja version = 3.1.2
  libyaml = True
COLLECTION VERSION
# /home/mark/PRIVATE/venv/lib/python3.10/site-packages/ansible_collections
Collection         Version
------------------ -------
azure.azcollection 1.13.0 

# /home/mark/.ansible/collections/ansible_collections
Collection         Version
------------------ -------
azure.azcollection 1.13.0 

CONFIGURATION
ANSIBLE_NOCOWS(env: ANSIBLE_NOCOWS) = True
CALLBACKS_ENABLED(/home/mark/PRIVATE/ansible.cfg) = ['profile_tasks']
DEFAULT_LOG_PATH(/home/mark/PRIVATE/ansible.cfg) = /home/mark/ansible.log
DEPRECATION_WARNINGS(/home/mark/PRIVATE/ansible.cfg) = False
STEPS TO REPRODUCE
  1. Create a keyvault
  2. Give set secret rights to service principal PRIVATE
  3. ansible localhost -m azure_rm_keyvaultsecret -a 'secret_name=somesecret secret_value=somevalue keyvault_uri=https://SOMEURL.vault.azure.net/ subscription_id=PRIVATE tenant=PRIVATE client_id=PRIVATE secret=PRIVATE'
EXPECTED RESULTS

The secret is set.

ACTUAL RESULTS

I get this error. Note that 04b07795-8ddb-461a-bbee-02f9e1bf7b46 is not the app id that I specified (or that I know about).

The user, group or application 'appid=04b07795-8ddb-461a-bbee-02f9e1bf7b46;oid=480ab1d9-a1a7-4a53-a509-15185d58d9a0;numgroups=1;iss=https://sts.windows.net/PRIVATE/' does not have secrets set permission on key vault 'PRIVATE;location=centralus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287\n",

(Note that private information below has be replaced)

ansible [core 2.13.5]
  config file = /home/mark/PRIVATE/ansible.cfg
  configured module search path = ['/home/mark/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mark/PRIVATE/venv/lib/python3.10/site-packages/ansible
  ansible collection location = /home/mark/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/mark/PRIVATE/venv/bin/ansible
  python version = 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
  jinja version = 3.1.2
  libyaml = True
Using /home/mark/PRIVATE/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Loading callback plugin minimal of type stdout, v2.0 from /home/mark/PRIVATE/venv/lib/python3.10/site-packages/ansible/plugins/callback/minimal.py
redirecting (type: callback) ansible.builtin.profile_tasks to ansible.posix.profile_tasks
Loading collection ansible.posix from /home/mark/PRIVATE/venv/lib/python3.10/site-packages/ansible_collections/ansible/posix
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.
META: ran handlers
redirecting (type: modules) ansible.builtin.azure_rm_keyvaultsecret to azure.azcollection.azure_rm_keyvaultsecret
Loading collection azure.azcollection from /home/mark/.ansible/collections/ansible_collections/azure/azcollection
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: mark
<127.0.0.1> EXEC /bin/sh -c 'echo ~mark && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mark/.ansible/tmp `"&& mkdir "` echo /home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144 `" && echo ansible-tmp-1666970046.6247046-1261657-10193813758144="` echo /home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144 `" ) && sleep 0'
redirecting (type: modules) ansible.builtin.azure_rm_keyvaultsecret to azure.azcollection.azure_rm_keyvaultsecret
Using module file /home/mark/.ansible/collections/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py
<127.0.0.1> PUT /home/mark/.ansible/tmp/ansible-local-1261651jh9_c5qm/tmpkewc945b TO /home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/ /home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/home/mark/PRIVATE/venv/bin/python /home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py", line 107, in <module>
    _ansiballz_main()
  File "/home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_keyvaultsecret', init_globals=dict(_module_fqn='ansible_collections.azure.azcollection.plugins.modules.azure_rm_keyvaultsecret', _modlib_path=modlib_path),
  File "/usr/lib/python3.10/runpy.py", line 209, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py", line 268, in <module>
  File "/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py", line 264, in main
  File "/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py", line 140, in __init__
  File "/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 469, in __init__
  File "/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py", line 184, in exec_module
  File "/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py", line 252, in create_update_secret
  File "/home/mark/PRIVATE/venv/lib/python3.10/site-packages/azure/keyvault/key_vault_client.py", line 1586, in set_secret
    raise models.KeyVaultErrorException(self._deserialize, response)
azure.keyvault.models.key_vault_error.KeyVaultErrorException: (Forbidden) The
user, group or application 'appid=04b07795-8ddb-461a-bbee-02f9e1bf7b46;oid=480ab1d9-a1a7-4a53-a509-15185d58d9a0;numgroups=1;iss=https://sts.windows.net/PRIVATE/' does not have secrets set permission on key vault 'PRIVATE;location=centralus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287
localhost | FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/mark/.ansible/tmp/ansible-tmp-1666970046.6247046-1261657-10193813758144/AnsiballZ_azure_rm_keyvaultsecret.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_keyvaultsecret', init_globals=dict(_module_fqn='ansible_collections.azure.azcollection.plugins.modules.azure_rm_keyvaultsecret', _modlib_path=modlib_path),\n  File \"/usr/lib/python3.10/runpy.py\", line 209, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.10/runpy.py\", line 96, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py\", line 268, in <module>\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py\", line 264, in main\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py\", line 140, in __init__\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py\", line 469, in __init__\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py\", line 184, in exec_module\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_46z9e_6k/ansible_azure_rm_keyvaultsecret_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_keyvaultsecret.py\", line 252, in create_update_secret\n  File \"/home/mark/PRIVATE/venv/lib/python3.10/site-packages/azure/keyvault/key_vault_client.py\", line 1586, in set_secret\n    raise models.KeyVaultErrorException(self._deserialize, response)\nazure.keyvault.models.key_vault_error.KeyVaultErrorException: (Forbidden) The user, group or application 'appid=04b07795-8ddb-461a-bbee-02f9e1bf7b46;oid=480ab1d9-a1a7-4a53-a509-15185d58d9a0;numgroups=1;iss=https://sts.windows.net/PRIVATE/' does not have secrets set permission on key vault 'PRIVATE;location=centralus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
@markscottwright
Copy link
Contributor Author

markscottwright commented Oct 28, 2022

The same problem appears in these modules:

  • azure_rm_keyvaultkey_info
  • azure_rm_keyvaultkey
  • azure_rm_keyvaultsecret_info
  • azure_rm_keyvaultsecret

This fixes the issue (assuming you are using a virtual environment and it's in venv):

find venv/ -type f -name 'azure*.py' -exec sed -i "s/elif.self.module.params..auth_source...in...auto....cli.../elif (self.module.params['auth_source'] == 'cli' or (self.module.params['auth_source'] == 'auto' and self.credentials['client_id'] is None and self.credentials['secret'] is None)):/" {} \;

@Fred-sun
Copy link
Collaborator

@markscottwright You are welcome to provide PR to fix this problem. Thank you very much!

@Fred-sun Fred-sun added medium_priority Medium priority work in In trying to solve, or in working with contributors labels Oct 31, 2022
markscottwright pushed a commit to markscottwright/azure that referenced this issue Oct 31, 2022
When service principals are specified in the module arguments
(subscription_id, tenant, client_id, secret) and auth_type is auto,
azure_rm_keyvaultkey, azure_rm_keyvaultkey_info,
azure_rm_keyvaultsecret, azure_rm_keyvaultsecret_info first attempted to
use credentials from env and the disk and only use the specified
credentials if not found.

To reproduce, specify a service principal in `cloud-config-azure.ini`
and run:

    `ansible-test integration azure_rm_keyvaultsecret --allow-destructive`

The test will succeed.  Then, do an `az login` and re-run the above
command.  The test will fail.  Delete your cached credentials:

    `rm ~/.azure/msal_token_cache.json`

Run the test again. The test will succeed.

c.f. ansible-collections#1009
@Fred-sun Fred-sun added has_pr PR fixes have been made and removed work in In trying to solve, or in working with contributors labels Nov 1, 2022
@Fred-sun
Copy link
Collaborator

@markscottwright auth_source specifies the priority of obtaining the credentials in azure_rm_common. All modules are common (azure_rm_.py), while the azure_rm_keyvault module specifies that 'cli' and 'auto' are used to obtain the cli login configuration. Thanks!

@markscottwright
Copy link
Contributor Author

@Fred-sun Are you saying this is as designed? If auth_source is 'auto' and credentials are specified, but cli credentials are present, then specified credentials will be ignored and the cli credentials used?

If so, we can't use this module, since we do not want to use cli credentials (and only discovered this issue when running a playbook on localhost after I had run some unrelated commands using the az cli client). It means playbooks can't reliably be run in parallel (since they will pick up each other's credentials). And of course, it means that sensitive information is persisted to the disk. The auth_source behavior used in other modules seems much better.

@Fred-sun
Copy link
Collaborator

@markscottwright Maybe we'll follow the logic of azure_rm_common.py here as well. get_keyvault_client can be obtained using env, credentials_file or az login. Will this meet your needs? Thanks!

@markscottwright
Copy link
Contributor Author

My understanding of azure_rm_common.py is that it goes "module parameters -> env -> credential_file -> cli" (this is what the azure_rm_keyvaultsecret docs say too). We need to use module parameters. Credentials file and cli are of no interest (since they persist the secrets to disk). We could, if necessary, use env, but would definitely prefer that azure_rm_keyvaultsecret be consistent with the rest of the azure modules.

xuzhang3 pushed a commit that referenced this issue May 30, 2023
When service principals are specified in the module arguments
(subscription_id, tenant, client_id, secret) and auth_type is auto,
azure_rm_keyvaultkey, azure_rm_keyvaultkey_info,
azure_rm_keyvaultsecret, azure_rm_keyvaultsecret_info first attempted to
use credentials from env and the disk and only use the specified
credentials if not found.

To reproduce, specify a service principal in `cloud-config-azure.ini`
and run:

    `ansible-test integration azure_rm_keyvaultsecret --allow-destructive`

The test will succeed.  Then, do an `az login` and re-run the above
command.  The test will fail.  Delete your cached credentials:

    `rm ~/.azure/msal_token_cache.json`

Run the test again. The test will succeed.

c.f. #1009

Co-authored-by: Mark Wright <mark.wright@primekey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
has_pr PR fixes have been made medium_priority Medium priority
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants