Supporting conda install in kedro install command

Description

Frequently in data science projects we need to use conda install to handle packages that require configuration / building steps after installation.

Context

I am currently doing a data science project in Kedro which uses deep learning. In this situation using conda install is important as it allows me to handle CUDA/cuDNN dependencies within the ecosystem.

This enhancement fits naturally as most Kedro installations use conda to create virtual environments anyway.

Possible Implementation

If someone creates a file called requirements_conda.txt in the src folder the following code should work to install via pip and conda.

@cli.command()
def install():
    """Install project dependencies from requirements.txt."""
    python_call("pip", ["install", "-U", "-r", "src/requirements.txt"])
    os.system('conda install --file src/requirements_conda.txt --yes')

I’d be happy to contribute the code myself, but I was wondering if this is something that you all envisioned supporting?

[ashwineepandey]

Hi, Not sure this issue has been resolved. I have seen the change that has been made. But I still get the following error when I run kedro install with an environment.yml file:

And this is in alignment with https://github.com/conda/conda/issues/6827#issuecomment-365614464

I believe instead of running conda install… it needs to change to conda env create --file envname.yml.

[ravindra-siwach]

Following up on this issue (I am aware that the changes are already incorporated in master): I used the kedro install version that uses conda env update --file src/environment.yml --prune After this step, I got the message about activating the newly created environment. After this, however, kedro goes through the process of further processing the dependencies mentioned in requirements.txt. Is this the intended behaviour? As far as I understand those packages will anyway not be installed in the env created through the environment.yml file. For my purposes, I already included all the packages from requirements.txt in environment.yml beforehand. Deleting or renaming the requirements.txt file only leads to kedro complaining that the file can’t be found.

I am just wondering whether it would be better that kedro install stops after the new env is created from environment.yml and not process requirements.txt too? and perhaps also activate the newly created environment in the same step.

I am happy to make the relevant changes and raise a PR.