Some tests seem to be flaky and fail depending on Python version used

Some tests (e.g. for Modelsim) do file comparisons between a reference Makefile and the Makefile generated by the tool run.

For example, the reference Makefile contains:

PLUSARGS      ?= plusarg_bool=1 plusarg_int=42 plusarg_str=hello

When running test_modelsim, the generated Makefile will contain:

PLUSARGS      ?= plusarg_int=42 plusarg_bool=1 plusarg_str=hello

I was running Python 3.5.9. With a different version you might get a different ordering, as this depends on how dictionaries are handled.

[GCHQDeveloper560]

Your diagnosis was the problem in #132. Pull request #143 fixes the current tests. Avoiding assumptions about dict ordering is probably good for robustness in general, but it would be nice if the tests were a little more flexible about the order of things that don’t matter.

I’d also like a better solution for mocking than our current crowd of scripts that all do the same thing in mock_commands. If we weren’t trying to support Windows I’d make them all symlinks to a single script, but haven’t come up with a better cross platform solution.

[olofk]

Does this mean we can close the issue now? Agree with @GCHQDeveloper560 that we want a better long-term solution though. Especially with the mocking. I just implemented the first approach I could think of, but there are likely far better ways to accomplish this. It’s also a bit leaky since it uses the host’s make (and probably other tools) IIRC