[question] Consistency of settings in tools only packages

As of 2020/10/09, here are the settings contributing to package_id in pure utilities packages:

Utilities only packages	settings	settings contributing to package_id
7zip	“os”, “arch”, “compiler”	“os”, “arch”
autoconf	“os”, “arch”	“os”, “arch”
automake	“os”	“os”
b2 (portable)	“os”, “arch”	“os”, “arch”
b2 (standard)	“os”, “arch”	“os”, “arch”
cc65	“os”, “arch”, “compiler”, “build_type”	“os”, “arch” (not with Visual Studio), “compiler”, “build_type”
cccl	“compiler”	None
cmake	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
depot_tools	“os_build”	“os_build”
doxygen	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
flatc	“os_build”, “arch_build”	“os_build”, “arch_build”
ftjam	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”
gettext	“os_build”, “arch_build”, “compiler”	“os_build”, “arch_build”
gnulib	“os_build”, “arch_build”	“os_build”, “arch_build”
gperf	“os_build”, “arch_build”, “compiler”	“os_build”, “arch_build”
guetzli	“os”, “arch”, “compiler”	“os”, “arch”
jom	“os”	“os” (Windows only anyway)
kcov	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
lemon	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
lzip	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
m4	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
make	“os_build”, “arch_build”, “compiler”	“os_build”, “arch_build”
makefile-project-workspace-creator	“os”	“os”
meson	None	None
mozilla-build	“os_build”, “arch_build”	“os_build”, “arch_build”
msys2	“os_build”, “arch_build”	“os_build”, “arch_build”
nasm	“os_build”, “arch_build”, “compiler”	“os_build”, “arch_build”
ninja 1.9.x	“os_build”, “arch_build”, “compiler”	“os_build”, “arch_build”
ninja 1.10.x	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
nodejs	“os_build”: [“Windows”, “Linux”, “Macos”], “arch_build”: [“x86_64”]	“os_build”, “arch_build”
premake	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
ragel	“os”, “arch”, “compiler”	“os”, “arch”
re2c	“os”, “arch”, “build_type”, “compiler”	“os”, “arch”, “build_type”, “compiler”
scons	“os”	“os”
strawberryperl	“os_build”, “arch_build”	“os_build” (only Windows anyway), “arch_build”
swig	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler”, “build_type”
verilator	“os”, “arch”, “compiler”, “build_type”	“os”, “arch”, “compiler” (not the version), “build_type”
waf	“os_build”, “arch_build”	“os_build”, “arch_build”
winflexbison	“os”, “arch”, “compiler”, “build_type”	“os” (Windows only anyway), “arch”, “compiler”, “build_type”
yasm	“os_build”, “arch_build”, “compiler”	“os_build”, “arch_build”

Packages are not really consistent with each other. What should be the proper settings?

(I would like also to centralize discussions about settings in utilities packages in this thread)

[jgsogo]

Hi! We’ve been talking internally about this issue again, and these are the conclusions:

Note.- We are always talking about installers, packages that provide an executable that is used as a tool. If, at some point in time, that package provides a library and it is also used as a regular requirement it will no longer be considered a installer.

A bit of context

Conan v2.0 will likely be using the new approach with the two profiles/contexts (build and host) by default. Some details are to be defined, but it looks like it is the only way to handle tools properly as build_requires. Old os_build and arch_build seems not to be flexible enough and make recipes harder to reuse in host contexts.

As said above by @ohanar , typically a user will use build_type=Release in the build context, so for sure we need to provide those binaries. But, as @madebr has pointed out several times, sometimes it is useful/needed to debug a tool running in the build context (attach the running process and debug step by step).

Proposal

• We need to list all the required settings in the recipe like any other regular package: this is needed in order to generate the binary matching the configuration provided in the CLI

  settings = "os", "arch", "compiler", "build_type"
  

  conan install cmake/3.18.2@ -s build_type=Debug  --build=cmake  # Generates Debug binaries
  conan install cmake/3.18.2@ -s build_type=Release --build=cmake  # Generates Release binaries
  

  Of course, if the package is not compiling from sources, some settings are not needed (like any other regular package).

• When debugging a tool, usually one needs to switch from Debug to Release, it is not enough to be able to compile the package in Debug, we need to switch from one to the other and it is not affordable to compile the binary each time (if they have the same package_id, the binaries are overriding). That’s the reason we agree we should preserve the build_type in the package_id()

• We remove the compiler in the package_id(self) method, we think no one finds it useful to have 20 different versions of CMake just because the package_id is different for every different compiler version. One of the value propositions of Conan is that you can remove some combinatorial from the settings and reuse the same binary for different configurations.

  def package_id(self):
      del self.info.settings.compiler
  

Future

In ConanCenter we will generate and provide binaries only for the Release configuration of these tools, that movement will require two developments:

• we need to implement in the CI something like https://github.com/conan-io/conan-center-index/issues/2813 to be able to select the packages to compile. Basically, we will remove every package-ID corresponding to a Debug build (assuming the corresponding Release one is not raising a ConanInvalidConfiguration).
• recipes for these installers will include a line to declare that the Release package is compatible with the build_type=Debug. This will be achieved via compatible_packages(docs). By that time it should be a stable feature.

Hope this makes sense. Thanks for all the effort you are dedicating to the ecosystem! 🙌

[madebr]

My 2c: The package_Id of tools-only packages should include os, arch, compiler, build_type (where applicable) because they also might need to be debugged. These recipes should be as tools and c3i should know to only build Release+MT for them.