[question] What level of testing is "conan test" intended to verify?

I am trying to design a library that I wish to create a Conan package from and I’m slightly confused as to what conan test is supposed to verify. Is conan test supposed to run all of my automated tests for the library (like my “GTest” tests that verify the full library functionality)? Or is it simply supposed to verify that the package can be installed correctly and can be consumed by another project and nothing more?

• I’ve read the CONTRIBUTING guide.

[KerstinKeller]

We’ve stumbled upon this too, and think that it would be beneficial if the test_package was part of the actual Conan package.

Running the test_package for each installed package would many times be very overkill, not intended for most of the cases, only for debugging purposes.

It doesn’t need to be run when consuming a package, just when building it (eg --build missing)

I see three main advantages of test_package belonging to the recipe:

• If you do a --build outdated the test package would not have to be build everytime, because Conan would know if it has changed or not. (Currently it is run always, even if the recipe is not outdated - we have to work around this in our CI)
• If a consumer builds with --build missing and for whatever reason their package cannot properly be consumed, then the consumer would get an error when building their dependency, instead of when building their application.
• Even more importantly, a package where test_package has failed can be uploaded to CI! it’s in the cache, laying there, and Conan upload will pick it up.

From my point it would be great if test_package where part of the conan package!

[Minimonium]

I understand where it does run, I’m asking about the problem of it testing only the creator configuration, which is fairly basic considering that we allow changing a package layout basing on literally anything. There is an infinite number of configurations out there that we don’t check. @Morwenn suggested in Slack that it could be beneficial to have test_package be exported with the recipe and check package state for a consumer’s configuration.