question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[bug] Tools collect_libs discards suffixes

See original GitHub issue

When porting the libpng project to Conan v2, the CCI hook complained that I lied about the library name in self.cpp_info.libs, which was supposed to be ["png16.a"] (on Linux/Mac is libpng16.a).

The truth is, the libpng projects creates a relative symbolic link, named as libpng.a, so we have two files in libdirs[0]: libpng16.a and its symlink libpng.a.

When writing the recipe, I avoided collect_libs and preferred and real name, so I pointed as:

self.cpp_info.libs = ["png16"] 

However, our CCI hook has a post cpp_info step which checks if what we declared for libs is actually installed in libdirs, and it failed. When checking the hook script, it uses collect_libs to list those installed libraries and compare with the libraries names from cpp_info. So, the method collect_libs only found "png", which means, it found the symbolic link, but ignored the real library.

Maybe it’s a expected behavior, as the symbolic link makes life easier to find the library, but still, it could be customized like collect_libs(self, ignore_symlinks=True) so, it would list only real files.

Environment Details (include every applicable attribute)

  • Operating System+version: Linux and Mac confirmed
  • Compiler+version: N/A
  • Conan version: 1.51.3
  • Python version: 3.10.4 (also tested with 3.9)

Steps to reproduce (Include if Applicable)

from conan import ConanFile
from conan.tools.files import collect_libs, save, mkdir
import os

class FooConan(ConanFile):
    def package(self):
        lib_folder = os.path.join(self.package_folder, "lib")
        lib_name = os.path.join(lib_folder, "libpng16.a")
        mkdir(self, lib_folder)
        save(self, lib_name, "FOOBAR")
        os.symlink(lib_name, os.path.join(lib_folder, "libpng.a"))

    def package_info(self):
        collected_libs = collect_libs(self)
        self.output.info(f"***** COLLECTED LIBS: {collected_libs}")
        self.cpp_info.libs = collected_libs
        # XXX: Collect libs only finds the symlink
        assert ["png"] == collected_libs
conan create . foo/0.1.0@ -pr:b=default -pr:h=default

Logs (Executed commands with output) (Include/Attach if Applicable)

Read the last lines of this log, the collected libs is there.

% conan create . foo/0.1.0@ -pr:b=default -pr:h=default
[HOOK - conan-center.py] pre_export(): [DEPRECATED GLOBAL CPPSTD (KB-H001)] OK
[HOOK - conan-center.py] pre_export(): [REFERENCE LOWERCASE (KB-H002)] OK
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'name' attribute. (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'url' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'license' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'description' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'homepage' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'topics' attribute.  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H003) 
[HOOK - conan-center.py] pre_export(): WARN: [HEADER_ONLY, NO COPY SOURCE (KB-H005)] This recipe is a header only library as it does not declare 'settings'. Please include 'no_copy_source' to avoid unnecessary copy steps
[HOOK - conan-center.py] pre_export(): [HEADER_ONLY, NO COPY SOURCE (KB-H005)] OK
[HOOK - conan-center.py] pre_export(): [FPIC OPTION (KB-H006)] OK
[HOOK - conan-center.py] pre_export(): [VERSION RANGES (KB-H008)] OK
[HOOK - conan-center.py] pre_export(): [RECIPE FOLDER SIZE (KB-H009)] Total recipe size: 1.0537109375 KB
[HOOK - conan-center.py] pre_export(): [RECIPE FOLDER SIZE (KB-H009)] OK
[HOOK - conan-center.py] pre_export(): [EXPORT LICENSE (KB-H023)] exports: None
[HOOK - conan-center.py] pre_export(): [EXPORT LICENSE (KB-H023)] exports: None
[HOOK - conan-center.py] pre_export(): [EXPORT LICENSE (KB-H023)] OK
[HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H024) 
[HOOK - conan-center.py] pre_export(): [META LINES (KB-H025)] OK
[HOOK - conan-center.py] pre_export(): [CONAN CENTER INDEX URL (KB-H027)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE MINIMUM VERSION (KB-H028)] OK
[HOOK - conan-center.py] pre_export(): [TEST PACKAGE - RUN ENVIRONMENT (KB-H029)] OK
[HOOK - conan-center.py] pre_export(): [SYSTEM REQUIREMENTS (KB-H032)] OK
[HOOK - conan-center.py] pre_export(): [CONANDATA.YML FORMAT (KB-H030)] OK
[HOOK - conan-center.py] pre_export(): [TEST PACKAGE - NO IMPORTS() (KB-H034)] OK
[HOOK - conan-center.py] pre_export(): [NO AUTHOR (KB-H037)] OK
[HOOK - conan-center.py] pre_export(): [NOT ALLOWED ATTRIBUTES (KB-H039)] OK
[HOOK - conan-center.py] pre_export(): [NO TARGET NAME (KB-H040)] OK
[HOOK - conan-center.py] pre_export(): [NO FINAL ENDLINE (KB-H041)] OK
[HOOK - conan-center.py] pre_export(): [NO REQUIRES.ADD() (KB-H044)] OK
[HOOK - conan-center.py] pre_export(): [DELETE OPTIONS (KB-H045)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE VERBOSE MAKEFILE (KB-H046)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE VERSION REQUIRED (KB-H048)] OK
[HOOK - conan-center.py] pre_export(): [CMAKE WINDOWS EXPORT ALL SYMBOLS (KB-H049)] OK
[HOOK - conan-center.py] pre_export(): [DEFAULT OPTIONS AS DICTIONARY (KB-H051)] OK
[HOOK - conan-center.py] pre_export(): [CONFIG.YML HAS NEW VERSION (KB-H052)] OK
[HOOK - conan-center.py] pre_export(): [PRIVATE IMPORTS (KB-H053)] OK
[HOOK - conan-center.py] pre_export(): [SINGLE REQUIRES (KB-H055)] OK
[HOOK - conan-center.py] pre_export(): [TOOLS RENAME (KB-H057)] OK
[HOOK - conan-center.py] pre_export(): [ILLEGAL CHARACTERS (KB-H058)] OK
[HOOK - conan-center.py] pre_export(): [CLASS NAME (KB-H059)] OK
[HOOK - conan-center.py] pre_export(): [NO CRLF (KB-H060)] OK
[HOOK - conan-center.py] pre_export(): [NO BUILD SYSTEM FUNCTIONS (KB-H061)] OK
[HOOK - conan-center.py] pre_export(): [TOOLS CROSS BUILDING (KB-H062)] OK
[HOOK - conan-center.py] pre_export(): [INVALID TOPICS (KB-H064)] OK
[HOOK - conan-center.py] pre_export(): [NO REQUIRED_CONAN_VERSION (KB-H065)] OK
[HOOK - conan-center.py] pre_export(): [TEST_TYPE MANAGEMENT (KB-H068)] OK
[HOOK - conan-center.py] pre_export(): [TEST PACKAGE - NO DEFAULT OPTIONS (KB-H069)] OK
[HOOK - conan-center.py] pre_export(): WARN: [MANDATORY SETTINGS (KB-H070)] No 'settings' detected in your conanfile.py. Add 'settings' attribute and use 'package_id(self)' method to manage the package ID.
[HOOK - conan-center.py] pre_export(): [MANDATORY SETTINGS (KB-H070)] OK
Exporting package recipe
[HOOK - conan-center.py] post_export(): [CONANDATA.YML REDUCE (KB-H031)] OK
[HOOK - conan-center.py] post_export(): [DEFAULT SHARED OPTION VALUE (KB-H050)] OK
[HOOK - conan-center.py] post_export(): [LICENSE PUBLIC DOMAIN (KB-H056)] OK
foo/0.1.0: The stored package has not changed
foo/0.1.0: Using the exported files summary hash as the recipe revision: 618bfcde05f3183716ed6832be4463a6 
foo/0.1.0: Exported revision: 618bfcde05f3183716ed6832be4463a6
Configuration (profile_host):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=13
os=Macos
os_build=Macos
[options]
[build_requires]
[env]
[conf]

Configuration (profile_build):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=13
os=Macos
os_build=Macos
[options]
[build_requires]
[env]
[conf]

foo/0.1.0: Forced build from source
Installing package: foo/0.1.0
Requirements
    foo/0.1.0 from local cache - Cache
Packages
    foo/0.1.0:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Build

Installing (downloading, building) binaries...
foo/0.1.0: Copying sources to build folder
foo/0.1.0: Building your package in /Users/uilian/.conan/data/foo/0.1.0/_/_/build/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
foo/0.1.0: Generator txt created conanbuildinfo.txt
foo/0.1.0: Aggregating env generators
[HOOK - conan-center.py] pre_build(): [FPIC MANAGEMENT (KB-H007)] 'fPIC' option not found
[HOOK - conan-center.py] pre_build(): [FPIC MANAGEMENT (KB-H007)] OK
foo/0.1.0: Calling build()
foo/0.1.0: WARN: This conanfile has no build step
foo/0.1.0: Package '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9' built
foo/0.1.0: Build folder /Users/uilian/.conan/data/foo/0.1.0/_/_/build/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
foo/0.1.0: Generated conaninfo.txt
foo/0.1.0: Generated conanbuildinfo.txt
foo/0.1.0: Generating the package
foo/0.1.0: Package folder /Users/uilian/.conan/data/foo/0.1.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
foo/0.1.0: Calling package()
[HOOK - conan-center.py] post_package(): ERROR: [PACKAGE LICENSE (KB-H012)] No 'licenses' folder found in package: /Users/uilian/.conan/data/foo/0.1.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9  (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H012) 
[HOOK - conan-center.py] post_package(): [DEFAULT PACKAGE LAYOUT (KB-H013)] OK
[HOOK - conan-center.py] post_package(): ERROR: [MATCHING CONFIGURATION (KB-H014)] Package for Header Only does not contain artifacts with these extensions: ['h', 'h++', 'hh', 'hxx', 'hpp'] (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H014) 
[HOOK - conan-center.py] post_package(): ERROR: [MATCHING CONFIGURATION (KB-H014)] Packaged artifacts does not match the settings used: os=None, compiler=None (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H014) 
[HOOK - conan-center.py] post_package(): [SHARED ARTIFACTS (KB-H015)] OK
[HOOK - conan-center.py] post_package(): [PC-FILES (KB-H020)] OK
[HOOK - conan-center.py] post_package(): [CMAKE-MODULES-CONFIG-FILES (KB-H016)] OK
[HOOK - conan-center.py] post_package(): [PDB FILES NOT ALLOWED (KB-H017)] OK
[HOOK - conan-center.py] post_package(): [LIBTOOL FILES PRESENCE (KB-H018)] OK
[HOOK - conan-center.py] post_package(): [MS RUNTIME FILES (KB-H021)] OK
[HOOK - conan-center.py] post_package(): [SHORT_PATHS USAGE (KB-H066)] OK
[HOOK - conan-center.py] post_package(): [MISSING SYSTEM LIBS (KB-H043)] OK
foo/0.1.0 package(): Packaged 2 '.a' files: libpng16.a, libpng.a
foo/0.1.0: Package '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9' created
foo/0.1.0: Created package revision fceead34a403492c2294e026d9a318fa
foo/0.1.0: ***** COLLECTED LIBS: ['png']
[HOOK - conan-center.py] post_package_info(): [CMAKE FILE NOT IN BUILD FOLDERS (KB-H019)] OK
[HOOK - conan-center.py] post_package_info(): [LIBRARY DOES NOT EXIST (KB-H054)] OK
[HOOK - conan-center.py] post_package_info(): ERROR: [INCLUDE PATH DOES NOT EXIST (KB-H071)] Component foo::foo include dir 'include' is listed in the recipe, but not found in package folder. The include dir should probably be fixed or removed. (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H071) 

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
memshardedcommented, Aug 24, 2022

In libpng recipe cpp_info.libs can be set to png[d] (or libpng[d] if msvc), which I think is neater anyway.

I think that if a recipe has some relatively simple logic that makes explicit the name of the final libraries, that is much preferred over a collect_libs. I know it is more effort on the recipe creator side, but following the python zen, explicit is better than implicit.

1reaction
SpaceImcommented, Aug 24, 2022

The hook is useful, but indeed https://github.com/conan-io/conan/pull/11527 broke it since it was relying on previous behavior of collect_libs where everything with .lib, .so, .a, .dylib extensions was collected (it’s worth noting that this previous behavior didn’t collect .so.<version> files). The hook should be updated to still being able to find all files under the libdirs, and be happy if libs in cpp_info.libs match at list one of those files.

In libpng recipe cpp_info.libs can be set to png[d] (or libpng[d] if msvc), which I think is neater anyway.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Tools — conan 1.46.2 documentation
If vcvars modifies an environment variable by appending values to the old value (separated by ; ), only the new values will be...
Read more >
PyInstaller Documentation - Read the Docs
community support, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. PyInstaller is an free software project that is ...
Read more >
apache-arrow | Yarn - Package Manager
Apache Arrow in JS. npm version. Arrow is a set of technologies that enable big data systems to process and transfer data quickly....
Read more >
Source - GitHub
Apache Arrow 6.0.1 (2021-11-18) ## Bug Fixes ... [C++][Python] Expose method for benchmarking tools to release unused memory from the allocators ...
Read more >
Apache Arrow 6.0.1 (2021-11-10)
ARROW-13492 - [R] [CI] Move r tools 35 build back to per-commit/pre-PR ... ARROW-11066 - [Java] Is there a bug in flight AddWritableBuffer ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found