Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] /usr/bin/salt-call state.apply hangs at end on Xubuntu 20.04 salt 3001

See original GitHub issue

Description Running /usr/bin/salt-call state.apply on Xubuntu 20.04 minion finishes but never completes.

Summary for local
--------------
Succeeded: 153
Failed:      0
--------------
Total states run:     153
Total run time:    16.030 s

But command never returns back to terminal.

ps -ef |grep salt-call
root     2928868 2914767  0 08:12 pts/28   00:00:00 sudo /usr/bin/salt-call state.apply
root     2928869 2928868  0 08:12 pts/28   00:00:05 /usr/bin/python3 /usr/bin/salt-call state.apply
root     3026694 3026554  0 09:37 pts/27   00:00:00 grep --color=auto salt-call

Notice the command has been running over an hour but only took 16 seconds for the states to be applied.

Setup

Steps to Reproduce the behavior Xubuntu 20.04 install with ZFS root

more /etc/apt/sources.list.d/saltstack.list
deb [arch=amd64] http://repo.saltstack.com/py3/ubuntu/20.04/amd64/latest focal main

Salt master is Ubuntu 18.04 version 3001 recently upgraded from nigrogen. All other minions on network are 18.04 and working as expected.

Expected behavior salt-call state.apply to finish and return to terminal.

Screenshots

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Master:
salt --versions-report
Salt Version:
           Salt: 3001
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: unknown
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.8
         Jinja2: 2.10
        libgit2: 0.26.0
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: Not Installed
   pycryptodome: 3.4.7
         pygit2: 0.26.2
         Python: 3.6.9 (default, Apr 18 2020, 01:56:04)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 17.1.2
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5
 
System Versions:
           dist: ubuntu 18.04 Bionic Beaver
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-101-generic
         system: Linux
        version: Ubuntu 18.04 Bionic Beaver

Minion With issue:
Salt Version:
           Salt: 3001
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.7.3
      docker-py: 4.1.0
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: 1.1.0
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: 3.6.1
         pygit2: Not Installed
         Python: 3.8.2 (default, Apr 27 2020, 15:53:34)
   python-gnupg: 0.4.5
         PyYAML: 5.3.1
          PyZMQ: 18.1.1
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.2
 
System Versions:
           dist: ubuntu 20.04 focal
         locale: utf-8
        machine: x86_64
        release: 5.4.0-39-generic
         system: Linux
        version: Ubuntu 20.04 focal


Working Minon:

salt-call --versions-report
Salt Version:
           Salt: 2017.7.4
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.9 (default, Apr 18 2020, 01:56:04)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5
 
System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-101-generic
         system: Linux
        version: Ubuntu 18.04 bionic

Additional context Add any other context about the problem here.

Issue Analytics

State:
Created 3 years ago
Comments:37 (20 by maintainers)

Top GitHub Comments

3reactions

sagetheragecommented, Aug 13, 2020

@ocfmatt we are working on a long term solution, but since we do not maintain this library it is taking some time. It was one suggestion to try to downgrade, but we realize that isn’t really a good idea, either, but looking for a real fix to this, still.

2reactions

mgdotsoncommented, Jul 3, 2020

I’m hesitant on #57356 as well as I keep most of my states without jinja references and filter via grains in the top.sls file.

It also fails if I run just a single state file. For example, my vscode state:

salt-call state.apply vscode

cat vscode/init.sls 
vscode_repo:
  pkgrepo.managed:
    - name: deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main
    - file: /etc/apt/sources.list.d/vscode.list
    - key_url: https://packages.microsoft.com/keys/microsoft.asc
    - refresh_db: true

vscode:
  pkg.installed:
    - name: code
    - require:
        - pkgrepo: vscode_repo

Output is similar to other fails:

local:
----------
          ID: vscode_repo
    Function: pkgrepo.managed
        Name: deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main
      Result: True
     Comment: Package repo 'deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main' already configured
     Started: 13:02:08.425043
    Duration: 107.504 ms
     Changes:   
----------
          ID: vscode
    Function: pkg.installed
        Name: code
      Result: True
     Comment: All specified packages are already installed
     Started: 13:02:09.400992
    Duration: 73.534 ms
     Changes:   

Summary for local
------------
Succeeded: 2
Failed:    0
------------
Total states run:     2
Total run time: 181.038 ms
^C

CTRL-C after waiting a couple of minutes