[QUESTION] How to update log file path based on test result

I’ve been playing around with the logging and so far it’s great! But now I am wanting to do a bit more customization to suite our needs.

So basically we have a lot of API tests to get through and we’re logging both PASSED and FAILED assertions to our logs and the HTML report.

This part is fine, except for the small hiccup that any one particular log file could be thousands of lines long.

So I have created a couple of functions to help split these logs on a per-test basis.

This is to get the current test details:

def get_current_test():
    path_to_module_and_test_name = os.environ['PYTEST_CURRENT_TEST'].split(' ')[0]
    list_details = path_to_module_and_test_name.split("::")
    path_to_module = list_details[0]
    test_name_with_data = list_details[1]
    path_details_list = path_to_module.split('/')
    test_module = path_details_list[-1]

    service_name = 'general'

    for x in ast.literal_eval(os.environ['API_SERVICES']):
        if x in path_details_list:
            service_name = x
            break

    if '[' in test_name_with_data:
        test_name = test_name_with_data.split('[')[0]
        test_data = test_name_with_data.replace(test_name, '')
    else:
        test_name = test_name_with_data
        test_data = test_name_with_data

    os.environ['test_module'] = test_module
    os.environ['test_service'] = service_name
    os.environ['test_name'] = test_name
    os.environ['test_data'] = test_data

Then I have a fixture to ‘remove and create’ the logger:

@pytest.fixture(autouse=True)
def update_logger():
    # get test details
    get_current_test()

    # set default status
    test_status = 'UNKNOWN'

    # prevent double logs
    logger.remove()

    # set logging
    logger.add(Path(os.environ.get('reports_dir'),
                    test_status,
                    os.environ['test_service'],
                    f'{os.environ["test_name"]}.log'),
               format=formatter)
    logger.add(sys.stderr, format=formatter)

And so far this works good for us. We end up with a break-down like this:

Reports
└── 18.03.2022
    └── 15.38.54
        ├── Execution_Report.html
        └── general
            └── test_again.log

And so basically, the idea is that we’re able to split log files on a per test basis (or whatever we want really) to keep the logs as short and precise as possible.

BUT, this approach introduces a lot of overhead for having to look through MANY directories and log files to find the logs for failed tests.

So I searched a bit and there’s a way to grab the result of a test, like so:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    # execute all other hooks to obtain the report object
    outcome = yield
    rep = outcome.get_result()

    # we only look at actual [PASS/FAIL/SKIP] test calls, not setup/teardown
    if rep.when == "call":
        if rep.passed:
            test_status = 'PASS'
        elif rep.failed:
            test_status = 'FAIL'
        elif rep.skipped:
            test_status = 'SKIP'
        else:
            test_status = 'ERROR'

And from this, I’d like to update the path that the logs would be generated in.

So I’d end up with something like this:

Reports
└── 18.03.2022
    └── 15.38.54
        ├── Execution_Report.html
        ├── FAIL
        │   └── general
        │       └── test_again.log
        └── PASS
            └── general
                └── Execution_Log.log

But the problem is that the path is already set for the log file in the update_logger fixture and I’m wondering if there’s a way of changing that path location in the pytest_runtest_makereport fixture?

[Delgan]

That looks fine. 😉

Here is a slightly simplified version to explain the principle in broad terms:

from loguru import logger
import pytest
import os
import shutil


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    rep = outcome.get_result()

    if rep.when == "call":
        if rep.passed:
            test_status = 'PASS'
        elif rep.failed:
            test_status = 'FAIL'
        elif rep.skipped:
            test_status = 'SKIP'
        else:
            test_status = 'ERROR'

        os.makedirs(test_status, exist_ok=True)
        shutil.move(item.session.config._logfile_path, test_status)

@pytest.fixture(autouse=True)
def update_logger(request):
    logfile_path = "logs/" + request.node.name + ".log"
    request.config._logfile_path = logfile_path
    logger.remove()
    handler_id = logger.add(logfile_path, mode="w")
    yield
    logger.remove(handler_id)

Note I’m using pytest.Config to store the log filepath and move it ad the end of the test (once it has been removed by fixture).

[Delgan]

Great, glad I could help you. 👍

I’m closing this issue then. 😉