ResolvedContext.get_environ() is missing key / values from parent_environ

ResolvedContext.get_environ() doesn’t return parent information, even if a variable is explicitly set via context.get_environ(self, parent_environ={"FOO": "bar"}). This behavior differs from execute_command(..., parent_environ={"FOO": "bar"}) and execute_shell(parent_environ={"FOO": "bar"}), which both include FOO in the resulting resolve.

Environment

• OS: CentOS 7
• Rez version: 2.109.0
• Rez python version: 3.7

To Reproduce Make a Python file like this:

from rez import resolved_context

requests = []
context = resolved_context.ResolvedContext(requests)
environment = context.get_environ(parent_environ={"FOO": "bar})

Expected behavior For FOO to be included in the environment variable, just like how doing export FOO=bar && rez-env -- echo '$FOO' prints bar.

Actual behavior environment does not include FOO unless one of the package requests in requests modifies it.

At the moment I’m getting around the problem by doing this

environment = {"FOO": "bar"}
environment.update(context.get_environ(parent_environ=environment))

Rez should be managing the environment variables, so I find this hack to be a bit fragile. I’m not sure if it is “guaranteed” to work and would rather Rez handle it for me.

[nerdvegas]

To be absolutely clear:

• get_environ() returns only the vars that rez packages in the resolve have altered. This is because rez doesn’t ‘care’ about any others - they remain unchanged
• parent_environ is there so you can supply an environ dict different to current os.environ. If None, os.environ is used (as per docstring).

So, to get the exact environ that you’d have in the rez-env, where you also want to set FOO, you would actually do this:

environment = os.environ.copy()
environment.update({"FOO": "bar"})
environment.update(context.get_environ(parent_environ=environment))

This will give you the environ equivalent to if you had done

]$ export FOO=bar && rez-env ...

One more subtlety - get_environ works by running the context through the “python” interpreter (a special non-shell case). That means that if any packages used the source function, and that caused an env-var to be set, that would not be picked up here (because there’s no shell to source the script!).

I think it would be worth updating the docstring to make this more clear. Ticket: https://github.com/AcademySoftwareFoundation/rez/issues/1317

Ps - Tbh I’m not really aware of get_environ being used much, we never used it at Method to my knowledge. If you wanted a 100% reliable version of this function (ie exactly what a shell would give), that would be something different - you’d have to optionally specify the shell to use, and a standard function would have to be added to every shell impl to be able to extract the resulting environ dict from it. In this version, we would have no choice other than to return the full environ, because it’s impossible to know what a source command might have set.

On Wed, Jun 8, 2022 at 1:17 AM Colin Kennedy @.***> wrote:

If the env-vars that potentially impact the environ are important to you, then manually combining them as per your second example is the correct approach.

The important thing in this case is to know the environment variables that would exist, via API, as if the user had called rez-env (for the first time) from the terminal. FWIW in my previous reply, I wasn’t implying that any of the behavior I’d mentioned were bugs, as you say they’re by design. And on several occasions that environment variable passing has been useful (even for nested rez-env calls) so I’d definitely like to that feature to stay.

So just to confirm, the conclusion on this issue is “get_environ() doesn’t show parent environment variables because if it did then you wouldn’t know which environment variable came from a resolve or which was from the parent?” And the work-around is to

environment = {“FOO”: “bar”}environment.update(context.get_environ(parent_environ=environment))

And I can be sure that this will always be consistent? If the answer is yes then I guess I’ll go forward with that. But if there’s any doubt that the above may not work, I’d suggest we add a new parameter to get_environ(), maybe include_parent=True or something so Rez can manage and return in the case where I, as the API caller, don’t care which variables were set via context as long as the result is consistent with what I’d see from a rez-env.

— Reply to this email directly, view it on GitHub https://github.com/AcademySoftwareFoundation/rez/issues/1316#issuecomment-1148814237, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMOUSRMKKERNAPP7XENTCTVN5RZXANCNFSM5YBMX7XA . You are receiving this because you modified the open/close state.Message ID: @.***>

[ColinKennedy]

If you wanted a 100% reliable version of this function (ie exactly what a shell would give), that would be something different

Maybe in the future but FWIW my case for get_environ() is for a debug tool, nothing fancy or critical. Accuracy is ideal but if we can get 99% of the way there and only source() or some other less common features aren’t supported, that’d be okay, I think. I’ll just add a disclaimer that the view may not show 100% everything. Thank you for the in-depth explanation!