Report tests results

Issue Type

• Feature request

Molecule and Ansible details

  config file = None
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/vagrant/vEnv27/lib/python2.7/site-packages/ansible
  executable location = /home/vagrant/vEnv27/bin/ansible
  python version = 2.7.5 (default, Aug  2 2016, 04:20:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]
molecule, version 2.22rc3

Molecule installation method :

any

Ansible installation method (one of):

• pip

Desired Behavior

I need to display a report of failed test and for each, the reason of failure. I didn’t see any option to do it in molecule. Even in proposed related issues/features.

In my environment, i will be use tox. I found pytest-molecule but it seems to focus only in infra test and not other molecule tests (as idempotence for example).

Will I have missed something ? If no, I think it might be a good improvement. The final goal is to have something like cucumber report in Jenkins but the first step will be to have the tests results generated somewhere.

Thanks

[seandst]

In addition to pytest’s ability to write out junit xml when running the testinfra tests, ansible has the junit callback plugin that can export the results of a playbook run to JUnit xml: https://docs.ansible.com/ansible/latest/plugins/callback/junit.html

I do recommend working with Ansible 2.8 or higher, as the “test_case_prefix” param to the callback plugin is great for only including specific tasks in the output for test reporting.

[ssbarnea]

I will close this because there are already few solutions (pytest and junit callback) and it is very unlikely that we will want to make a reporting tool out of molecule. That reminds me of the issues pytest-coverage went into due to the decision to include reporting. Now they struggle to take it out.