Package gets installed into build_env instead of host_env when cross-compiling

Hi @wolfv, I am observing the following weird bug in RoboStack when trying to cross-build pcl-conversions for osx-arm64.

The recipe.yaml looks as follows:

(base) tobias@Tobiass-MacBook-Pro ros-noetic % cat recipe.yaml 
package:
  name: ros
  version: 2021.05.21

source:
  - git_url: https://github.com/ros-gbp/perception_pcl-release.git
    git_rev: release/noetic/pcl_conversions/1.7.2-1
    folder: ros-noetic-pcl-conversions/src/work

build:
  number: 0

about:
  home: https://www.ros.org/
  license: BSD-3-Clause
  summary: |
    Robot Operating System

extra:
  recipe-maintainers:
    - ros-forge

outputs:
  - package:
      name: ros-noetic-pcl-conversions
      version: 1.7.2
    requirements:
      build:
        - "{{ compiler('cxx') }}"
        - "{{ compiler('c') }}"
        - ninja
        - sel(unix): make
        - cmake
        - sel(build_platform != target_platform): python
        - sel(build_platform != target_platform): cross-python_{{ target_platform }}
        - sel(build_platform != target_platform): cython
      host:
        - eigen
        - pcl 1.11
        - python
        - ros-distro-mutex 0.1 noetic
        - ros-noetic-catkin
        - ros-noetic-pcl-msgs
        - ros-noetic-roscpp
        - ros-noetic-sensor-msgs
        - ros-noetic-std-msgs
        - vtk
      run:
        - eigen
        - pcl
        - python
        - ros-distro-mutex 0.1 noetic
        - ros-noetic-pcl-msgs
        - ros-noetic-roscpp
        - ros-noetic-sensor-msgs
        - ros-noetic-std-msgs
        - vtk
        - sel(osx and x86_64): __osx >={{ MACOSX_DEPLOYMENT_TARGET|default('10.14') }}
    build:
      script:
        sel(win): bld_catkin.bat
        sel(unix): build_catkin.sh

Note that vtk shows up in the build and run dependencies.

I get an error that:

CMake Error at /Users/tobias/opt/anaconda3/conda-bld/ros_1621565257168/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/cmake/vtk-9.0/VTKPython-targets.cmake:392 (message):
  The imported target "VTK::vtkCommonCorePython" references the file

     "/Users/tobias/opt/anaconda3/conda-bld/ros_1621565257168/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.8/site-packages/vtkmodules/vtkCommonCore.so"

When activating the host_env, vtk does not show up in mamba list. However, it does show up in the build_env:

# packages in environment at /Users/tobias/opt/anaconda3/conda-bld/ros_1621565257168/_build_env:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                hc929b4f_4    conda-forge
c-ares                    1.17.1               h0d85af4_1    conda-forge
ca-certificates           2020.12.5            h033912b_0    conda-forge
catkin                    0.8.10                   pypi_0    pypi
catkin-pkg                0.4.23                   pypi_0    pypi
cctools_osx-64            949.0.1             h6407bdd_22    conda-forge
cctools_osx-arm64         949.0.1             h55de48a_22    conda-forge
certifi                   2020.12.5        py38h50d1736_1    conda-forge
clang                     11.1.0               h694c41f_1    conda-forge
clang-11                  11.1.0          default_he082bbe_1    conda-forge
clang_osx-arm64           11.1.0               h40be2b1_2    conda-forge
clangxx                   11.1.0          default_he082bbe_1    conda-forge
clangxx_osx-arm64         11.1.0               hc50fe25_2    conda-forge
cmake                     3.20.2               hdbd2f3e_1    conda-forge
compiler-rt               11.1.0               h654b07c_0    conda-forge
compiler-rt_osx-64        11.1.0               h8c5fa43_0    conda-forge
cross-python_osx-arm64    3.8                  11_cpython    conda-forge
crossenv                  1.0                pyhd8ed1ab_1    conda-forge
cython                    0.29.23          py38ha048514_0    conda-forge
docutils                  0.17.1                   pypi_0    pypi
empy                      3.3.4                    pypi_0    pypi
expat                     2.3.0                he49afe7_0    conda-forge
gencpp                    0.6.5                    pypi_0    pypi
geneus                    3.0.0                    pypi_0    pypi
genlisp                   0.4.18                   pypi_0    pypi
genmsg                    0.5.16                   pypi_0    pypi
gennodejs                 2.0.2                    pypi_0    pypi
genpy                     0.6.15                   pypi_0    pypi
gettext                   0.19.8.1          h7937167_1005    conda-forge
krb5                      1.19.1               hcfbf3a7_0    conda-forge
ld64_osx-64               530                 he8994da_21    conda-forge
ld64_osx-arm64            530                 h3681fde_21    conda-forge
ldid                      2.1.2                h7660a38_2    conda-forge
libclang-cpp11.1          11.1.0          default_he082bbe_1    conda-forge
libcurl                   7.76.1               hf45b732_2    conda-forge
libcxx                    12.0.0               habf9029_0    conda-forge
libedit                   3.1.20191231         hed1e85f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libffi                    3.3                  h046ec9c_2    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
libllvm11                 11.1.0               hd011deb_2    conda-forge
libllvm12                 12.0.0               hd011deb_1    conda-forge
libnghttp2                1.43.0               h07e645a_0    conda-forge
libssh2                   1.9.0                h52ee1ee_6    conda-forge
libuv                     1.41.0               hbcf498f_0    conda-forge
llvm-tools                11.1.0               hd011deb_2    conda-forge
loguru                    0.5.3                    pypi_0    pypi
lz4-c                     1.9.3                h046ec9c_0    conda-forge
make                      4.3                  h22f3db7_1    conda-forge
ncurses                   6.2                  h2e338ed_4    conda-forge
ninja                     1.10.2               h9a9d8cb_0    conda-forge
nose                      1.3.7                    pypi_0    pypi
openssl                   1.1.1k               h0d85af4_0    conda-forge
pip                       21.1.1             pyhd8ed1ab_0    conda-forge
popt                      1.16              h112b980_2002    conda-forge
pyparsing                 2.4.7                    pypi_0    pypi
python                    3.8.10          h3a889e0_1_cpython    conda-forge
python-dateutil           2.8.1                    pypi_0    pypi
python_abi                3.8                      1_cp38    conda-forge
pyyaml                    5.4.1                    pypi_0    pypi
readline                  8.1                  h05e3726_0    conda-forge
rhash                     1.4.1                h35c211d_0    conda-forge
rsync                     3.2.3                h0d3f7e0_2    conda-forge
sed                       4.7               h3efe00b_1000    conda-forge
sensor-msgs               1.13.1                   pypi_0    pypi
setuptools                49.6.0           py38h50d1736_3    conda-forge
six                       1.16.0                   pypi_0    pypi
sqlite                    3.35.5               h44b9ce1_0    conda-forge
tapi                      1100.0.11            h879752b_0    conda-forge
tk                        8.6.10               hb0a8c7a_1    conda-forge
vtk                       9.0.1                    pypi_0    pypi
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
xxhash                    0.8.0                h35c211d_3    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
zlib                      1.2.11            h7795811_1010    conda-forge
zstd                      1.5.0                h582d3a0_0    conda-forge

It seems to be specific to vtk, the other dependencies (including pcl) are correctly installed in the host_env (although note the sensor-msgs. It happens both locally on a Mac as well as in the CI.

[wolfv]

I think these lines from the cross-python activation script might be relevant:

https://github.com/conda-forge/cross-python-feedstock/blob/41083676420cef1f476a0bde41632cdf7334279b/recipe/activate-cross-python.sh#L33-L34

Maybe @isuruf has an idea (sorry to bother you!). I think for us the problem is that we need to link to the shared libs in site-packages.

However we don’t try to execute these python executables so there won’t be any issues with executing the wrong arch… maybe we could improve this activation script in a way to not remove some dylib or so files depending on some configured env var …

@Tobias-Fischer we should probably move this issue to the cross-python feedstock

[wolfv]

I am closing this as we debugged this as a cross-python behavior and there is not much we can do with boa (except implementing what cross-python does at a boa level … but maybe that’s a different discussion?!).