- Save HashiCorp's Vault secrets as json/yaml dump
- Populate specific Vault prefix from json/yaml dump
You can read how to create scheduled backup with Jenkins in this post: Scheduled backup of Vault secrets with Jenkins on Kubernetes
The second part of the 'Scheduled backup of HashiCorp's Vault secrets', using Kubernetes CronJob to create scheduled backups on daily basis post: https://igorzhivilo.com/vault/scheduled-backup-vault-cronjob/
- Dump all your secrets as encrypted yaml/json files, or you can store it without encryption.
- Get a list of your secrets.
- Print all secrets nicely.
- Populate Vault from yaml/json dumps to a specific 'vault_prefix'.
How can an application programmatically request a token so that it can read secrets from Vault?
Using the AppRole which is an authentication mechanism within Vault to allow machines or apps to acquire a token to interact with Vault and using the policies you can set access limitations for your app.
It uses RoleID and SecretID for login.
Read more about AppRole authentication: https://learn.hashicorp.com/vault/identity-access-management/approle https://igorzhivilo.com/jenkins/how-to-read-vault-secrets-from-declarative-pipeline/
$ vault auth enable approle
$ vault secrets enable kv-v2
I run vault inside of Kubernetes(EKS) cluster, UI not publically available that why I use port-forwarding
$ kubectl port-forward -n vault svc/vault 8200
Forwarding from 127.0.0.1:8200 -> 8200
Forwarding from [::1]:8200 -> 8200
Go to policy tab -> Create ACL Policy (jenkins policy in this example)
path "sys/auth/approle" {
capabilities = [ "create", "read", "update", "delete", "sudo" ]
}
path "sys/auth/approle/*" {
capabilities = [ "create", "read", "update", "delete" ]
}
path "auth/approle/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
path "sys/policies/acl/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
path "secret/data/jenkins/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
path "secret/metadata/jenkins/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
click 'create' this policy
attach to jenkins approle -> jenkins policy (using CLI)
$ vault write auth/approle/role/jenkins token_policies=jenkins \
token_ttl=1h token_max_ttl=4h
$ vault read auth/approle/role/jenkins/role-id
$ vault write -f auth/approle/role/jenkins/secret-id
vault write auth/approle/login \
role_id=ROLE_ID \
secret_id=SECRET_ID
- VAULT_ADDR: for example: 'http://vault.vault.svc.cluster.local:8200' for k8s cluster
- ROLE_ID: RoleID for AppRole auth
- SECRET_ID: SecretID for AppRole auth
- VAULT_PREFIX: for example 'jenkins'
- ENCRYPTION_KEY: encryption key(used by Fernet library) to encrypt your secrets dump
export VAULT_ADDR=http://vault.vault.svc.cluster.local:8200
export ROLE_ID=7ae17fff9-11aa-16d2-b5cc-dbb964a50917
export SECRET_ID=6ae17eee9-11aa-16d2-b5cc-aaa964a99917
export VAULT_PREFIX=jenkins
export ENCRYPTION_KEY="21UEdPvS7JJJBsfWbSoNM3h5AP1IowZ7iUP_YQWVoqo=",
pip install -r requirements.txt
$ python vault_handler.py
Specify one of the commands below
print
print-dump
dump
populate
$ python vault_handler.py print
$ python vault_handler.py dump --help
Usage: vault_handler.py dump [OPTIONS]
: Dump secrets from Vault (Dump will be encrypted).
Options:
-dp, --dump_path TEXT Path/name of dump with secrets
--help Show this message and exit.
# encrypted dump of secrets will be created: vault_secrets.enc
$ python vault_handler.py dump
python vault_handler.py populate --help
Usage: vault_handler.py populate [OPTIONS]
: Populate Vault prefix from dump with secrets.
Options:
-vp, --vault_prefix TEXT Vault's prefix to populate from secrets dump
[required]
-dp, --dump_path TEXT Path to dump with secrets
--help Show this message and exit.
python vault_handler.py populate -vp 'test_prefix' -dp 'vault_secrets.enc'
- support for different authentication mechanisms
- add to PyPI as package
- example of how to run scheduled backup from Jenkins (Look at examples/jenkins/Jenkinsfile).
- example of how to run scheduled backup using vault_handler and k8s cronjob (Look at examples/cronjob/cronjob.yaml).
PRs with improvements/features are welcomed!