cocotb 1.4 issue in scoreboard with hexdiffs and hexdumps

Since release 1.4 where strings passed to cocotb.utils.hexdump() and cocotb.utils.hexdiffs() are deprecated, I found out that I experienced issues with the scoreboard results when tests fail. Although all my tests are updated to work with bytes instead of string data, it seems that inside the scoreboard code the got and exp data is still internally converted to strings before the cocotb.utils.hexdump() and cocotb.utils.hexdiffs() are called. It results in a hanging behaviour of the simulator. The same applies for the log.info() where expected and received data is logged inside this piece of code.

[eric-wieser]

If we want to support rich comparison behavior we would need to introduce a comparison API for all values going into scoreboard.

This feels like a problem we should delegate to pytest, which already has a fair number of comparison tools, source here:

[ktbarrett]

@hjwmedenblik @eric-wieser The use of str is causing serious problems. Passing bytes to str() returns the representation, it does not decode the string. The resulting string is not equivalent to the original bytes, for example b'\x00\x01\x02\x03' becomes the string "b'\x00\x01\x02\x03'" which has 19 characters. Your exp value as a string is 6083 characters long. This is not only incorrect, but explodes that O(n**2) algo and makes it seem to hang.

We need to fix the scoreboard. I actually have no idea what the intent of the code in the scoreboard is. I’m guessing it was only supposed to ever be used with BinaryValue, but even then hexdiffs isn’t appropriate since BinaryValue.__str__ never returned a hex string. My inclination is to just change the scoreboard to do simpler reporting like I suggested above. But I also think that @hjwmedenblik should just stop using the scoreboard. I remember we talked on the gitter about how I was trying to get rid of the testbenching components because they were “fake useful”, I think you understand now.