test_closedown fails on s390x architecture with Icarus

(I’m not actually sure this is a cocotb issue, but I thought I’d report it here anyway, just to be sure).

I’ve been maintaining cocotb packages in Fedora’s repositories. Before 1.4, the package was a “noarch” package-- that is to say, it didn’t contain any compiled code, since the simulator libraries had to be built by the user at runtime. I’ve finally gotten around to updating to cocotb 1.4; as part of that I’ve transitioned the package to an arched package-- with a build for each supported Fedora architecture, with the libraries pre-built. (We build Icarus Verilog on all architectures, so in theory, I think it makes sense to have builds on any architecture where iverilog is available).

To make sure that things work, from inside the Fedora buildsystem I run the test suite against Icarus. All the tests seem to pass on all the architectures… except for s390x, where one fails:

**Failure in testsuite: 'all' classname: 'test_closedown' testcase: 'test_failure_from_system_task' with parameters 'all'**

The logs from running test_closedown:

/usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s close_module -f sim_build/cmds.f -g2012   /builddir/build/BUILD/cocotb-1.4.0/tests/test_cases/test_closedown/../../../tests/designs/close_module/close_module.v
MODULE=test_closedown TESTCASE= TOPLEVEL=close_module TOPLEVEL_LANG=verilog \
        /usr/bin/vvp -M /builddir/build/BUILDROOT/python-cocotb-1.4.0-3.fc34.s390x/usr/lib64/python3.9/site-packages/cocotb/libs -m libcocotbvpi_icarus   sim_build/sim.vvp 
     -.--ns INFO     cocotb.gpi                         ..mbed/gpi_embed.cpp:74   in set_program_name_in_venv        Did not detect Python virtual environment. Using system-wide Python interpreter
     -.--ns INFO     cocotb.gpi                         ../gpi/GpiCommon.cpp:105  in gpi_print_registered_impl       VPI registered
     -.--ns INFO     cocotb.gpi                         ..mbed/gpi_embed.cpp:244  in embed_sim_init                  Python interpreter initialized and cocotb loaded!
     0.00ns INFO     cocotb                                      __init__.py:202  in _initialise_testbench           Running on Icarus Verilog version 10.3 (stable)
     0.00ns INFO     cocotb                                      __init__.py:208  in _initialise_testbench           Running tests with cocotb v1.4.0 from /builddir/build/BUILDROOT/python-cocotb-1.4.0-3.fc34.s390x/usr/lib64/python3.9/site-packages/cocotb
     0.00ns INFO     cocotb                                      __init__.py:229  in _initialise_testbench           Seeding Python random module with 1599234468
     0.00ns INFO     cocotb.regression                         regression.py:127  in __init__                        Found test test_closedown.test_failure_from_system_task
     0.00ns INFO     cocotb.regression                         regression.py:127  in __init__                        Found test test_closedown.test_after_system_task_fail
     0.00ns INFO     cocotb.regression                         regression.py:459  in _start_test                     Running test 1/2: test_failure_from_system_task
     0.00ns INFO     ..re_from_system_task.0x3ffb50d8fd0       decorators.py:255  in _advance                        Starting test: "test_failure_from_system_task"
                                                                                                                     Description: 
                                                                                                                         Allow the dut to call system tasks from verilog.
                                                                                                                         $fatal() will fail the test, and scheduler will cleanup forked coroutines.
                                                                                                                         
10000000.00ns ERROR    cocotb.regression                         regression.py:374  in _score_test                     Test passed but we expected a failure: test_failure_from_system_task (result was TestSuccess)
10000000.00ns INFO     cocotb.regression                         regression.py:459  in _start_test                     Running test 2/2: test_after_system_task_fail
10000000.00ns INFO     ..er_system_task_fail.0x3ffb5056f70       decorators.py:255  in _advance                        Starting test: "test_after_system_task_fail"
                                                                                                                       Description: 
                                                                                                                           Test to run after failed test.
                                                                                                                           
10000001.00ns INFO     cocotb.regression                         regression.py:361  in _score_test                     Test Passed: test_after_system_task_fail
10000001.00ns ERROR    cocotb.regression                         regression.py:475  in _log_test_summary               Failed 1 out of 2 tests (0 skipped)
10000001.00ns INFO     cocotb.regression                         regression.py:548  in _log_test_summary               ******************************************************************************************************
                                                                                                                       ** TEST                                          PASS/FAIL  SIM TIME(NS)  REAL TIME(S)  RATIO(NS/S) **
                                                                                                                       ******************************************************************************************************
                                                                                                                       ** test_closedown.test_failure_from_system_task    FAIL     10000000.00         25.12    398019.58  **
                                                                                                                       ** test_closedown.test_after_system_task_fail      PASS            1.00          0.00      8857.59  **
                                                                                                                       ******************************************************************************************************
                                                                                                                       
10000001.00ns INFO     cocotb.regression                         regression.py:565  in _log_sim_summary                *************************************************************************************
                                                                                                                       **                                 ERRORS : 1                                      **
                                                                                                                       *************************************************************************************
                                                                                                                       **                               SIM TIME : 10000001.00 NS                         **
                                                                                                                       **                              REAL TIME : 25.13 S                                **
                                                                                                                       **                        SIM / REAL TIME : 397903.49 NS/S                         **
                                                                                                                       *************************************************************************************

So… it seems like $fatal() didn’t exit the simulator. Unfortunately, it’s hard to know just from the log what exactly has happened here-- is this an issue with iverilog, or the cocotb interface to iverilog? This test passes on all the other architectures Fedora supports, but s390x is the only big endian architecture we support, so… that could be related.

I could see if I can get shell access to a s390x system through Fedora to try and debug this further (I suppose just seeing if $fatal works from Verilog code without cocotb would be a simple enough thing to try), or if there’s something I can do through cocotb I’m happy to try that. But I wanted to file an issue somewhere so there was at least a record of the problem.

[eric-wieser]

We would also accept a PR that just skips that test on that platform, I think.

[ktbarrett]

The logs you posted seem to indicate that the $finish() task is never called by the HDL. I think we will need @bluecmd’s assistance tracking down the root cause. Is it our registering of the task that doesn’t work? Or is is Icarus’s calling of it that doesn’t occur / doesn’t work?