Question about conda-meta exclusionSee original GitHub issue
I notice that I can create a non-base conda environment that includes
conda>4.5, pack this environment, and later unpack it without issue.
conda create -n pack-base python=3 conda pip # normal conda output removed for brevity ... conda pack -n pack-base --arcroot pack-base Collecting packages... Packing environment at '/Users/parente/miniconda3/envs/pack-base' to 'pack-base.tar.gz' [########################################] | 100% Completed | 5.0s # on another host tar xzf pack-base.tar.gz -C tmp source /tmp/pack-base/bin/activate conda unpack
I notice that
conda-meta is not included by default in the packed environment, which means
conda list (and maybe other conda commands?) does not function after unpacking.
conda list EnvironmentLocationNotFound: Not a conda environment: /tmp/pack-base
However, if I manually add the
conda-meta from the original environment to the tarball so that it’s available later,
conda seems to work properly after unpacking. So much so that I can list, install, uninstall, etc. packages in that unpacked environment.
Could you explain why
conda-meta is not included in the tarball generated by
conda pack? I’m guessing there’s a good reason, and that if I add
conda-meta to my packed envs, I’ll hit some surprise down the line that I’ve not discovered yet.
I ask because I have an interest in packing and distributing environments in which
conda itself functions.
- Created 5 years ago
Top GitHub Comments
Ah, good to know (I should really read the conda release notes). In this case, I think the todo list for this functionality is:
conda-metafrom the environment (looks like new conda doesn’t put a
/binunless conda was explicitly installed in the environment.
conda-metato fix the absolute path and link information
Thanks for looking in to this!
Apologies for the delay here, still catching up after vacation.
conda-metadirectory (and a few other conda-specific things) were stripped from the environment because the intent of this project was to distribute conda environments to locations where conda wasn’t already installed (my use case specifically was deployment on YARN). Since this doesn’t handle the root environment, and conda can only be installed in the root environment, this can’t be used to distribute environments that contain conda. And if a machine already has conda installed on it, it seems cleaner and more robust to try and recreate the environment using something like
That said, I don’t see any harm in not stripping out
conda-meta. The file sizes are small and should compress well, and users can always filter it out if needed. A few potential issues that come to mind (untested):
We also strip out
/bin, and replace
deactivatewith our own versions. This may break workflows using standard conda tools, or it may not. Unsure.
conda-metafiles include a couple absolute paths (mostly package cache information). I think these are harmless to copy over (conda is fairly robust to malformed package caches), but the best thing to do would probably be to fix
link.typein all files to be
pathtypefor each path record), and replace the absolute paths with empty strings or some bogus string (to strip the user information).
I don’t have time currently to test these issues out, but if you do @parente I’d accept and review a PR.