`test_no_empty_file_items` not working on some CI setups

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

This is a bit weird. The test spyder/plugins/findinfiles/widgets/tests/test_widgets.py::test_no_empty_file_items has started failing on the Debian CI system (which uses lxc containers). Some of these failures can be seen in the test logs at https://ci.debian.net/packages/s/spyder/unstable/amd64/ Here’s a relevant part of one of them:

=================================== FAILURES ===================================
___________________________ test_no_empty_file_items ___________________________

findinfiles = <spyder.plugins.findinfiles.widgets.main_widget.FindInFilesWidget object at 0x7f15887e4280>
qtbot = <pytestqt.qtbot.QtBot object at 0x7f1588f8beb0>

    @flaky(max_runs=5)
    @pytest.mark.skipif(not running_in_ci(), reason="Only works on CIs")
    @pytest.mark.skipif(os.name == 'nt', reason="Fails on Windows")
    def test_no_empty_file_items(findinfiles, qtbot):
        """
        Test that a search that hits the max number of results doesn't generate
        empty file items.
    
        This is a regression test for issue spyder-ide/spyder#16256
        """
        findinfiles.set_search_text("spam")
        findinfiles.set_directory(osp.join(LOCATION, "data"))
        findinfiles.set_conf('max_results', 6)
    
        with qtbot.waitSignal(findinfiles.sig_max_results_reached):
            findinfiles.find()
    
        # Assert the results are the expected ones to reproduce the bug this test
        # tries to catch. In other words, this is here to prevent future changes to
        # our test files that would render this test useless.
        results = {
            'spam.py': [(2, 7), (5, 1), (7, 12)],
            'spam.txt': [(1, 0), (1, 5), (3, 22)]
        }
>       assert process_search_results(findinfiles.result_browser.data) == results
E       AssertionError: assert {'spam.cpp': [(2, 9), (6, 15), (8, 2)], 'spam.py': [(2, 7), (5, 1), (7, 12)]} == {'spam.py': [(2, 7), (5, 1), (7, 12)], 'spam.txt': [(1, 0), (1, 5), (3, 22)]}
E         Common items:
E         {'spam.py': [(2, 7), (5, 1), (7, 12)]}
E         Left contains 1 more item:
E         {'spam.cpp': [(2, 9), (6, 15), (8, 2)]}
E         Right contains 1 more item:
E         {'spam.txt': [(1, 0), (1, 5), (3, 22)]}
E         Full diff:
E         - {'spam.py': [(2, 7), (5, 1), (7, 12)], 'spam.txt': [(1, 0), (1, 5), (3, 22)]}
E         + {'spam.cpp': [(2, 9), (6, 15), (8, 2)], 'spam.py': [(2, 7), (5, 1), (7, 12)]}

/tmp/autopkgtest-lxc.3rwfusl7/downtmp/build.Dr5/src/spyder/plugins/findinfiles/widgets/tests/test_widgets.py:340: AssertionError
=============================== warnings summary ===============================

But on my local setup, all works as expected, running what is supposedly an identical lxc container. The only difference I can think of is in the locale (I’m in en_GB.UTF-8, while I’m guessing the ci.debian.net ones are en_US.UTF-8), but trying to run an en_US.UTF-8 container locally failed to reproduce the problem. It seems that what has happened here is that in some setups, this find process is looking at the files in a different order from in other setups. But it is still returning exactly 6 finds. I don’t know whether the order of file searching is deterministic across locales/setups, but I’m guessing that may be the source of this test failure.

Versions

• Spyder version: 5.3.3
• Python version: 3.10.6
• Qt version:
• PyQt version:
• Operating System name/version: Debian/Linux testing

[ccordoba12]

I think for now we should try to add the missing cases to the test. As you @dalthviz, I’m worried that trying to sort the search results could impact negatively the performance of this plugin.

[juliangilbey]

Although it’s probably worth noting that if sticking with the current list-all-possibilities scheme, the assert can be simplified to:

    assert process_search_results(findinfiles.result_browser.data) in results