Salt minion didn't decrypt GPG encrypted pillar when using SaltClass #54145

pavel-z1 · 2019-08-08T12:24:16Z

Description of Issue

I have installed Salt with enabled SaltClass module.
Salt minion didn't decrypt GPG encrypted pillar.

Setup

Configured parameter in /etc/salt/master.d/f_defaults.conf

master_tops:
    saltclass:
      path: /srv/salt/saltclass

Configured Node Yaml file /srv/salt/saltclass/nodes/server1.yml:

---
environment: base

states:
  - cpanel.ips

classes:
  - roles.pass

Content of file /srv/salt/saltclass/classes/roles/pass.yml:

#!jinja|yaml|gpg

pillars:
  settings:
    crypted_secret: |
      -----BEGIN PGP MESSAGE-----
      Version: GnuPG v2.0.22 (GNU/Linux)

      hQEMA4Ip7oKHIdrwAQf6A5taDQqjoaKZ48cqFRXRNIwoCBGhC5ucQPulZDq+MKlu
      2iBhyKPWmiFZdLo2cB5liGYwoBooRUhoeInfIj/TxBSlOgnXPz+8aPA+wMvp2wOa
      uQ4kQPjiRfWXaKBONqG4OOWcM82LiZu205TEmWRnLyiQv9pmQmddwnr1Mm4GS4cV
      dhv4YpmFjiBXmQkoFizIAjsyjtWnjmc+m51X8tca18125GBrIoU0XDBfN7vcneV9
      hSKp7NKUZaJQjTQ7jBD9ePjAg/jO2iVpPWaou4cW1fDqxsNKRHyoKiV0855IB3O0
      ethZa5CKV6iu4XUOor7+baM3TuadtiZfMDGRJ+87fdJLASgbCSIzWGONnXiUCpub
      F5+j2PGuUzSforNNzplnGwY60+ZYOkNnhaS5xtFa9HIcdR3GCIBpS4DV5qQmuC5p
      RgZbJXIh0pKCGCnB
      =IlpW
      -----END PGP MESSAGE-----

Steps to Reproduce Issue

salt server1 pillar.get settings:crypted_secret
server1:
-----BEGIN PGP MESSAGE-----
Version: GnuPG v2.0.22 (GNU/Linux)

hQEMA4Ip7oKHIdrwAQf6A5taDQqjoaKZ48cqFRXRNIwoCBGhC5ucQPulZDq+MKlu
2iBhyKPWmiFZdLo2cB5liGYwoBooRUhoeInfIj/TxBSlOgnXPz+8aPA+wMvp2wOa
uQ4kQPjiRfWXaKBONqG4OOWcM82LiZu205TEmWRnLyiQv9pmQmddwnr1Mm4GS4cV
dhv4YpmFjiBXmQkoFizIAjsyjtWnjmc+m51X8tca18125GBrIoU0XDBfN7vcneV9
hSKp7NKUZaJQjTQ7jBD9ePjAg/jO2iVpPWaou4cW1fDqxsNKRHyoKiV0855IB3O0
ethZa5CKV6iu4XUOor7+baM3TuadtiZfMDGRJ+87fdJLASgbCSIzWGONnXiUCpub
F5+j2PGuUzSforNNzplnGwY60+ZYOkNnhaS5xtFa9HIcdR3GCIBpS4DV5qQmuC5p
RgZbJXIh0pKCGCnB
=IlpW
-----END PGP MESSAGE-----

Versions Report

Salt Master version:

salt --versions-report
Salt Version:
           Salt: 2018.3.4

Dependency Versions:
           cffi: 1.6.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: 0.26.3
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
   python-gnupg: 0.4.3
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.2.1511 Core
         locale: ANSI_X3.4-1968
        machine: x86_64
        release: 3.10.0-327.18.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.2.1511 Core

Salt minion version:
salt-call --versions-report

Salt Version:
           Salt: 2018.3.4

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
      docker-py: 1.10.6
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.31.0
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: 1.2.5
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.5 (default, Jun 24 2019, 08:54:15)
   python-gnupg: 0.4.3
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: redhat 7.6 Vladimir Lyakhov
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-962.3.2.lve1.5.25.8.el7.x86_64
         system: Linux
        version: CloudLinux 7.6 Vladimir Lyakhov

The text was updated successfully, but these errors were encountered:

max-arnold · 2019-08-08T16:03:16Z

Here is a workaround I use (it is slower than decrypting individual keys, but at least it works):

ext_pillar:
  - saltclass:
    - path: ./
  - gpg: {}

https://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.gpg.html#module-salt.pillar.gpg

/cc @a-a-abramov Possible feature idea for SaltClass

pavel-z1 · 2019-08-12T09:09:34Z

@max-arnold , Thanks a lot for the tip!
After adding this line to ext_pillar description started working:
- gpg: {}

dmurphy18 · 2019-08-12T19:28:28Z

@pavel-z1 Believe this was fixed by the following:

#50809
#50810

This should be available in Salt 2019.2.0. Can you retry with Salt 2019.2.0 release in your environment and report your results as to whether the fix was sufficient for your environment.

pavel-z1 · 2019-08-16T05:29:47Z

Hi @dmurphy18
I've checked with Salt master 2019.2.0 release.
Salt master 2019.2.0 has the same behavior.

With such ext_pillar configuration GPG pillar didn't decrypt:

ext_pillar:
  - saltclass:
    - path: /srv/salt/saltclass

With this configuration all works fine:

ext_pillar:
  - saltclass:
    - path: /srv/salt/saltclass
  - gpg:

Salt version:

salt --versions-report
Salt Version:
           Salt: 2019.2.0

Dependency Versions:
           cffi: 1.6.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: 0.26.3
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
   python-gnupg: 0.4.3
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.2.1511 Core
         locale: ANSI_X3.4-1968
        machine: x86_64
        release: 3.10.0-327.18.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.2.1511 Core

stale · 2020-01-08T04:10:21Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

max-arnold · 2020-01-08T05:37:23Z

Bump

stale · 2020-01-08T05:37:25Z

Thank you for updating this issue. It is no longer marked as stale.

dmurphy18 · 2020-01-14T18:10:57Z

@pavel-z1 Is this still an issue with Salt 2019.2.2 ?

stale · 2020-02-13T18:32:35Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

pavel-z1 · 2020-02-14T09:17:23Z

Hi @dmurphy18 ,

I've checked with latest version:

/usr/lib/python2.7/site-packages/salt/scripts.py:109: DeprecationWarning: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date.  Salt will drop support for Python 2.7 in the Sodium release or later.
Salt Version:
           Salt: 3000

Dependency Versions:
           cffi: 1.6.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.7.2
        libgit2: 0.26.3
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
   python-gnupg: 0.4.3
         PyYAML: 3.11
          PyZMQ: 15.3.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.2.1511 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-327.18.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.2.1511 Core

Without "- gpg:" in ext_pillar GPG still doesn't work.

stale · 2020-02-14T09:17:26Z

Thank you for updating this issue. It is no longer marked as stale.

stale · 2020-03-15T12:06:33Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

pavel-z1 · 2020-03-15T12:18:16Z

Issue is actual.
As I see, issue was added previously to the milestone.

stale · 2020-03-15T12:18:19Z

Thank you for updating this issue. It is no longer marked as stale.

sc-kanga · 2020-07-06T13:08:52Z

This is still an issue; is there no update from Salt? This seems rather lax.

I have replicated my setup on an Ubuntu dev environment which comes with a later version of gpg {{ gpg (GnuPG) 2.2.4
}}; however, the problem still exists unless you use the "--full-generate-key". Unfortunately this does not seem to be available on RHEL7.

Using the "--full-generate-key" option, you an add the /home/dir option which then - i haven't had the time to fully explore why, yet - seems to allow for pillar to decrypt the data. Again though; this only appears to be available with Ubuntu.

dmurphy18 · 2021-05-20T23:12:43Z

@pavel-z1 The main problem is related to not using ext_pillar in order to get saltclass rendered, as documented in https://docs.saltproject.io/en/latest/ref/pillar/all/salt.pillar.saltclass.html#saltclass-pillar-module.

taking your example of pass.sls but with the gpg issues removed and crypted_secret a text string 'dog'.
Without use of ext_pillar, nothing is rendered, with ext_pillar rending was correct.

root@Unknown:/srv/salt/saltclass# salt server1 pillar.items
server1:
    ----------
    __saltclass__:
        ----------
        classes:
            - roles.pass
        environment:
            base
        nodename:
            server1
        states:
            - cpanel.ips
    settings:
        ----------
        crypted_secret:
            dog

With #|jinja|yaml|gpg, rendering is

server1:
    ----------
    __saltclass__:
        ----------
        classes:
            - roles.pass
        environment:
            base
        nodename:
            server1
        states:
            - cpanel.ips
    settings:
        ----------
        crypted_secret:
            -----BEGIN PGP MESSAGE-----
            Version: GnuPG v2.0.22 (GNU/Linux)
            
            hQEMA4Ip7oKHIdrwAQf6A5taDQqjoaKZ48cqFRXRNIwoCBGhC5ucQPulZDq+MKlu
            2iBhyKPWmiFZdLo2cB5liGYwoBooRUhoeInfIj/TxBSlOgnXPz+8aPA+wMvp2wOa
            uQ4kQPjiRfWXaKBONqG4OOWcM82LiZu205TEmWRnLyiQv9pmQmddwnr1Mm4GS4cV
            dhv4YpmFjiBXmQkoFizIAjsyjtWnjmc+m51X8tca18125GBrIoU0XDBfN7vcneV9
            hSKp7NKUZaJQjTQ7jBD9ePjAg/jO2iVpPWaou4cW1fDqxsNKRHyoKiV0855IB3O0
            ethZa5CKV6iu4XUOor7+baM3TuadtiZfMDGRJ+87fdJLASgbCSIzWGONnXiUCpub
            F5+j2PGuUzSforNNzplnGwY60+ZYOkNnhaS5xtFa9HIcdR3GCIBpS4DV5qQmuC5p
            RgZbJXIh0pKCGCnB
            =IlpW
            -----END PGP MESSAGE-----

The SaltClass code has functions for render_yaml and render_jinja but none for render_gpg, presuming the need for gpg from Salt.
Also as part of the investigation I noticed that pillar.get did not return information but pillar.items does.
SaltClass has a large PR pending against it #42349, merged into develop branch but not master, and the code has moved on since, hence troublesome to port to the latest code.

SaltClass is part of the current process of moving some modules into salt.extensions, and investigating the porting the PR into the code after that process, with a lot of additional tests (pytest) the above issue with pillar.get will be investigated.

But as other have pointed out in the comments and documented, use ext_pillar.

If this explanation is satisfactory, please consider closing this issue, unless there is some other point to raise.

dmurphy18 · 2021-06-24T15:25:53Z

Closing this since working as expected. Please re-open if there is other information to consider or the explanation is unsatisfactory

dmurphy18 added this to the Approved milestone Aug 12, 2019

dmurphy18 added the info-needed waiting for more info label Aug 12, 2019

dmurphy18 self-assigned this Aug 12, 2019

stale bot added the stale label Jan 8, 2020

stale bot removed the stale label Jan 8, 2020

stale bot added the stale label Feb 13, 2020

stale bot removed the stale label Feb 14, 2020

stale bot added the stale label Mar 15, 2020

stale bot removed the stale label Mar 15, 2020

dmurphy18 added expected-behavior intended functionality and removed info-needed waiting for more info labels May 20, 2021

dmurphy18 closed this as completed Jun 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Salt minion didn't decrypt GPG encrypted pillar when using SaltClass #54145

Salt minion didn't decrypt GPG encrypted pillar when using SaltClass #54145

pavel-z1 commented Aug 8, 2019

max-arnold commented Aug 8, 2019

pavel-z1 commented Aug 12, 2019

dmurphy18 commented Aug 12, 2019

pavel-z1 commented Aug 16, 2019 •

edited

Loading

stale bot commented Jan 8, 2020

max-arnold commented Jan 8, 2020

stale bot commented Jan 8, 2020

dmurphy18 commented Jan 14, 2020

stale bot commented Feb 13, 2020

pavel-z1 commented Feb 14, 2020 •

edited

Loading

stale bot commented Feb 14, 2020

stale bot commented Mar 15, 2020

pavel-z1 commented Mar 15, 2020

stale bot commented Mar 15, 2020

sc-kanga commented Jul 6, 2020

dmurphy18 commented May 20, 2021

dmurphy18 commented Jun 24, 2021

Salt minion didn't decrypt GPG encrypted pillar when using SaltClass #54145

Salt minion didn't decrypt GPG encrypted pillar when using SaltClass #54145

Comments

pavel-z1 commented Aug 8, 2019

Description of Issue

Setup

Steps to Reproduce Issue

Versions Report

max-arnold commented Aug 8, 2019

pavel-z1 commented Aug 12, 2019

dmurphy18 commented Aug 12, 2019

pavel-z1 commented Aug 16, 2019 • edited Loading

stale bot commented Jan 8, 2020

max-arnold commented Jan 8, 2020

stale bot commented Jan 8, 2020

dmurphy18 commented Jan 14, 2020

stale bot commented Feb 13, 2020

pavel-z1 commented Feb 14, 2020 • edited Loading

stale bot commented Feb 14, 2020

stale bot commented Mar 15, 2020

pavel-z1 commented Mar 15, 2020

stale bot commented Mar 15, 2020

sc-kanga commented Jul 6, 2020

dmurphy18 commented May 20, 2021

dmurphy18 commented Jun 24, 2021

pavel-z1 commented Aug 16, 2019 •

edited

Loading

pavel-z1 commented Feb 14, 2020 •

edited

Loading