No top data for masterless minion with master_tops

[igorwidlinski]

We are using reclass adapter to generate our top and pillar data, this works well with salt master. Recently we found some use cases where we would like to run salt minion in masterless mode. We have it largely working, and we can use commends such as:

salt-call --local pillar.items
or
salt-call --local state.sls pki.host_certs (running single state that uses pillar data).

But we are unable to run state.highstate on the minion, because the minion is looking for top.sls and its either empty or doesn't exist. It does not seem to try to fetch top data using the reclass adapter. I can confirm its generating top data as expected as when i run salt-call in local mode I get the usual:

pillar:
  __reclass__
       ----------
                applications:
                    - apps here

Here is our minion config:

file_client: local
test: True
log_level: debug
rejected_retry: True
auth_tries: 10080
reclass: &reclass
    storage_type: yaml_fs
    inventory_base_uri: /srv/metadata
    reclass_source_path: /srv/salt-common/reclass

master_tops:
    reclass: *reclass

top_file_merging_strategy: merge

ext_pillar:
    - reclass: *reclass

pillar_opts: True

runner_dirs:
    - /srv/salt/salt-common/_runners


module_dirs:
    - /srv/salt-common/base/_modules/

file_roots:
  base:
    - /srv/salt-common/base
  dev:
    - /srv/salt-common/base/dev
    - /srv/salt-common/base

[jfindlay]

@igorwidlinski, thanks for the report. What is the output of salt-call --versions-report?

[igorwidlinski]

salt-call --versions-report
Salt Version:
Salt: 2015.8.0

Dependency Versions:
Jinja2: 2.2.1
M2Crypto: 0.20.2
Mako: Not Installed
PyYAML: 3.11
PyZMQ: 14.5.0
Python: 2.6.6 (r266:84292, Jul 22 2015, 16:47:47)
RAET: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: Not Installed
timelib: Not Installed

System Versions:
dist: redhat 6.3 Carbon
machine: x86_64
release: 2.6.32-573.3.1.el6.x86_64
system: Scientific Linux 6.3 Carbon

[igorwidlinski]

Its been about a month now. Is there any other info I can provide to help out?

[igorwidlinski]

Same issue with 2015.5.0

salt-minion --versions-report
           Salt: 2015.5.0
         Python: 2.6.6 (r266:84292, Jul 22 2015, 16:47:47)
         Jinja2: 2.2.1
       M2Crypto: 0.20.2
 msgpack-python: 0.1.13
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 4.0.4
           Mako: Not Installed

[jfindlay]

@igorwidlinski, your information is good. We have a large backlog of bugs to work on which is why no one has looked at this yet.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[sbreidba]

I'm attempting to use the new saltclass pillar/master_tops feature and am seeing similar behavior - I have a Windows minion, and am using salt-call --local, and don't get any topfile data from it. I tried adding a custom module to do some debugging, and it doesn't seem to get loaded. Am I doing something dumb, or is the master_tops system simply not available for master-less minions?

Salt Version:
           Salt: 2018.3.2

Dependency Versions:
           cffi: 1.10.0
       cherrypy: 10.2.1
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.3
          ioflo: Not Installed
         Jinja2: 2.9.6
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.6
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: 2.17
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)]
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.3
           RAET: Not Installed
          smmap: 2.0.3
        timelib: 0.2.4
        Tornado: 4.5.1
            ZMQ: 4.1.6

System Versions:
           dist:
         locale: cp1252
        machine: AMD64
        release: 10
         system: Windows
        version: 10 10.0.16299 SP0 Multiprocessor Free

[eric-anderton-at-sony]

For anyone else that wound up here, I may have figured out what's going on. The implementation of LocalClient (as of 2018.3.3) is deficient by design:

https://github.com/saltstack/salt/blob/v2018.3.3/salt/fileclient.py#L1041

    def master_tops(self):
        '''
        Originally returned information via the external_nodes subsystem.
        External_nodes was deprecated and removed in
        2014.1.6 in favor of master_tops (which had been around since pre-0.17).
             salt-call --local state.show_top
        ends up here, but master_tops has not been extended to support
        show_top in a completely local environment yet.  It's worth noting
        that originally this fn started with
            if 'external_nodes' not in opts: return {}
        So since external_nodes is gone now, we are just returning the
        empty dict.
        '''
        return {}

So you can't get there from here.

[max-arnold]

@Ch3LL Could you please reopen this issue? Just spent about 2 hours debugging the same problem, still relevant for 2019.2.0

[max-arnold]

Maybe it makes sense to have some specific label like pinned or backlog in the stale-bot exemptLabels for obvious feature deficiencies like this one (when a thing is expected to work but doesn't because it wasn't implemented yet)?

This could potentially reduce triaging efforts for duplicate tickets and requests to reopen an issue (like mine).

[stale]

Thank you for updating this issue. It is no longer marked as stale.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.