Sanity function to check for empty error output

I was creating a test in reframe, and hit a case where my sanity pattern was found in the output, yet there were also warnings going to stderr. Of course, if you expect a test to produce no warnings or errors, it makes a lot of sense to add a sanity check that checks if the error output is emtpy. I could however not find a default sanity test that does something like this…

Did I overlook it, or is it not implemented? If it isn’t implemented, would it be an idea (feature request coming up here) to add such a sanity function? It seems like a pretty generic case one may want to check.

I now did

@sn.sanity_function
def assert_iostream_empty(iostream):
    return sn.assert_not_found(r'.+', iostream)
...
self.sanity_patterns = sn.all([
    assert_iostream_empty(self.stderr),
    ...
])

This seems to work, but I can imagine there may be more elegant methods (and maybe a more low level implementation of checking for an emtpy file than a generic regex is more robust too).

[casparvl]

Ah cool, that is indeed more robust.

By the way, I now also print the first part of the error output

@sn.sanity_function
def assert_file_empty(file):
    try:
        if sn.assert_not_found(r'^$', file):
            return True
    except SanityError:
        with open(file, 'r') as errorfile:
            errMsg = errorfile.read()
        raise SanityError("File {0} not empty.\nFirst 300 chars:\n{1}".format(file, errMsg[0:300])) from None

For me, printing a few lines from the error output really helps to quickly identify why the sanity check fails, making it a bit faster than having to browse to the file.

Anyway, thanks for the improved regex suggestion. It all works for me now, I just figured I’d share the idea in case you think it can help more people and want to implement it as something more standardized 😃

[casparvl]

Yes, thanks! I will close it 😃