pillar.filter_by not working with salt-ssh

[sblaisot]

Description of Issue

pillar.filter_by seems to be broken under salt-ssh

It works correctly in a master-minion environment.

Steps to Reproduce Issue

• Add a pillar environment: dev
• Apply pillar.filter_by on a dict with this pillar

$ salt-ssh 'minion' pillar.get environment
minion:
    dev

$ salt-ssh 'minion' pillar.filter_by '{dev: aaa, prod: bbb, default: ccc}' environment
minion:
    ccc

Relevant debug logs are:

[DEBUG   ] Could not LazyLoad pillar.filter_by: 'pillar.filter_by' is not available.
[DEBUG   ] Performing shimmed, blocking command as follows:
pillar.filter_by {dev: aaa, int: bbb, default: ccc} environment
[DEBUG   ] Executing command: ssh minion -t -t -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  mkdir -p 
[DEBUG   ] Child Forked! PID: 27722  STDOUT_FD: 12  STDERR_FD: 14
[DEBUG   ] Terminal Command: /bin/sh -c ssh minion -t -t -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  mkdir -p 
[DEBUG   ] Executing command: scp -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  /tmp/shim_11rox8jk minion:.211fa71bf90d.py
[DEBUG   ] Child Forked! PID: 27728  STDOUT_FD: 12  STDERR_FD: 14
[DEBUG   ] Terminal Command: /bin/sh -c scp -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  /tmp/shim_11rox8jk minion:.211fa71bf90d.py
[DEBUG   ] Executing command: ssh minion -t -t -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  /bin/sh '$HOME/.211fa71bf90d.py'
[DEBUG   ] Child Forked! PID: 27732  STDOUT_FD: 12  STDERR_FD: 14
[DEBUG   ] Terminal Command: /bin/sh -c ssh minion -t -t -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  /bin/sh '$HOME/.211fa71bf90d.py'
[DEBUG   ] Executing command: ssh minion -t -t -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  rm '$HOME/.211fa71bf90d.py'
[DEBUG   ] Child Forked! PID: 27736  STDOUT_FD: 12  STDERR_FD: 14
[DEBUG   ] Terminal Command: /bin/sh -c ssh minion -t -t -o KbdInteractiveAuthentication=no -o PasswordAuthentication=no -o GSSAPIAuthentication=no -o ConnectTimeout=65 -o Port=22 -o IdentityFile=id_rsa -o User=root  rm '$HOME/.211fa71bf90d.py'
[DEBUG   ] RETCODE minion: 0
[DEBUG   ] SHIM retcode(0) and command: {
[DEBUG   ] LazyLoaded nested.output

on the same minion server, with a salt-minion connected to a salt-master and the same pillar:

$ salt-call pillar.filter_by '{dev: aaa, prod: bbb, default: ccc}' environment
local:
    aaa

Versions Report

Salt Version:
           Salt: 2019.2.3
 
Dependency Versions:
           cffi: 1.13.2
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10.3
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.9 (default, Nov  7 2019, 10:44:02)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 18.1.1
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.2
 
System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-76-generic
         system: Linux
        version: Ubuntu 18.04 bionic

[lkubb]

Stumbled upon this as well. Workaround by supplying a default key:

{%- set k = salt['pillar.get']('some:pillar', 'default') -%}
{%- set filtered = salt['pillar.filter_by'](somedict, 'some:pillar', default=k) -%}

I might debug this at some point if I find the time, this is quite confusing when it first happens.