Interdependent operations that depends on each other

Describe the bug Operations are planned according to the facts gathered at the start of the run. Operations usually changes the state of the host, this means that the preconditions that exists during the call to the operation is different that what is expected when declaring it.

To Reproduce

$ cat Dockerfile
FROM fedora:32

RUN dnf install -y git

$ docker build . -t my-fedora-image

$ cat deploy.py
from pyinfra import host
from pyinfra.operations import dnf

dnf.packages(
    packages=["git"],
    present=False,
    sudo=True,
)

dnf.packages(
    packages=["git"],
    present=True,
    sudo=True,
)

print(host.state.ops)
$ pyinfra --dry @docker/my-fedora-image deploy.py
--> Loading config...
--> Loading inventory...
    The @docker connector is in beta!

--> Connecting to hosts...
    [@docker/my-fedora-image] Connected

--> Preparing operations...
    Loading: deploy.py
{@docker/my-fedora-image: {'18996c2767e7142774328ef0b335b99342919a9e': {'commands': [StringCommand(dnf remove -y git)]}, '3b2f90227d43a40a2288506c2a99bdc5e71f32b2': {'commands': []}}}
    [@docker/my-fedora-image] Ready: deploy.py

--> Proposed changes:
    Groups: @docker
    [@docker/my-fedora-image]   Operations: 2   Commands: 1   
    [@docker/my-fedora-image] docker build complete, image ID: 39b782da6859

Expected behavior

$ pyinfra --dry @docker/my-fedora-image deploy.py
--> Loading config...
--> Loading inventory...
    The @docker connector is in beta!

--> Connecting to hosts...
    [@docker/my-fedora-image] Connected

--> Preparing operations...
    Loading: deploy.py
{@docker/my-fedora-image: {'18996c2767e7142774328ef0b335b99342919a9e': {'commands': [StringCommand(dnf remove -y git)]}, '3b2f90227d43a40a2288506c2a99bdc5e71f32b2': {'commands': [StringCommand(dnf install -y git)]}}}

    [@docker/my-fedora-image] Ready: deploy.py

--> Proposed changes:
    Groups: @docker
    [@docker/my-fedora-image]   Operations: 2   Commands: 2   
    [@docker/my-fedora-image] docker build complete, image ID: 39b782da6859

Meta

• pyinfra --support output

--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Linux
      Platform: Linux-5.4.0-40-generic-x86_64-with-glibc2.29
      Release: 5.4.0-40-generic
      Machine: x86_64
    pyinfra: v1.0.3
    Executable: /home/FooBarQuaxx/Code/pyinfra/venv/bin/pyinfra
    Python: 3.8.2 (CPython, GCC 9.3.0)

• installed from source
• no pyinfra-debug.log

[Fizzadar]

I am now confident that https://github.com/Fizzadar/pyinfra/pull/397 covers all of the cases listed here, which is excellent 😃

Need to decide next steps in terms of documenting and keeping track of any similar future issues, as they can be extremely nuanced/difficult to identify.

[Fizzadar]

@nikaro I believe this is failing due to fact collection (which happens before the user is created) is trying to use that user. In effect it’s not possible to create a user and then use them with the auth arguments sudo_user/su_user.

Just noticed your reddit reply which should indeed cover this, using the user/group arguments to files.* operations to avoid the need for su.

The auth arguments expect the users to already exist on the remote side. I need to a) document this better and b) replicate your issue and make it output something useful, because the error you’re getting is an awful experience!