Replace custom 'activate' method and symlink injection with a `MELTANO_PATH` env var - or a modified `PATH` - for child processes
See original GitHub issueIn #6468, we are improving our interop for how Meltano is able to create symlinks which ensure it will be made available to child processes and processes like Airflow which will want to callback or invoke Meltano directly.
Now that we have the EDK and a growing set of best practice paradigms, this seems like a problem we could solve in a better way than the symlink approach.
Two possible options for working around the symlink approach:
- Pass to child processes a
MELTANO_PATH
env var which points to the Meltano executable. - Before invoking a child process, prefix the child process’s
PATH
env var with the path to the Meltano executable.
These options also have the nice benefits of:
- Working well within containers, where Meltano is expected to be installed but may not be in an obvious location.
- Working well in contexts where multiple versions of Meltano might be installed.
Issue Analytics
- State:
- Created a year ago
- Reactions:2
- Comments:10 (6 by maintainers)
Top Results From Across the Web
How to set child process' environment variable in Makefile
The best way to prove that an environment variable has been set, is to query this environment variable inside another program that make...
Read more >hashicorp/envconsul: Launch a subprocess with ... - GitHub
Launch a subprocess with environment variables using data from @HashiCorp ... are given to the child process. custom = ["PATH=$PATH:/etc/myapp/bin"] # This ...
Read more >change environment of a running process - Unix & Linux Stack ...
Is there a way to change another process's environment variables? ... unique directory and pass the same path to the child shell script....
Read more >A process inherits its environment from its parent, and the ...
Of course, once the process is running, it is free to change its environment variables by calling SetEnvironmentVariable , and those modified ......
Read more >Env Variables and Modes - Vite
During production, these env variables are statically replaced. It is therefore necessary to always reference them using the full static string.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@aaronsteers The symlink existed only for the benefit of https://gitlab.com/meltano/files-airflow/-/blob/master/bundle/orchestrate/dags/meltano.py#L38, so indeed adding it to the
PATH
or exposing it throughMELTANO_PATH/EXECUTABLE
should be sufficient.(Since I/we don’t have a lot of background info here, perhaps good to phone a friend with more historical context 📞 😄.)
@DouweM - We postulate above that the symlink’s primary benefit is for subprocess/sideprocesses which need to find the meltano executable - and that similar functionality could be replaced by simply ensuring meltano is on the
PATH
or else explicitly pointed to in a futureMELTANO_PATH
env var which would be leveraged by meltano-integrated tools when it is found in context.By chance do you have any additional historic or technical context for the need for the symlink which we haven’t covered here?