question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Testing randomly fails

See original GitHub issue

I run ./scripts/test to check if everything is ok. Sometimes tests are passed, sometimes - no. This happens randomly.

Here are fails:

========================================================================= FAILURES ==========================================================================
_______________________________________________________________________ test_headers ________________________________________________________________________

    def test_headers():
        h = Headers(raw=[(b"a", b"123"), (b"a", b"456"), (b"b", b"789")])
        assert "a" in h
        assert "A" in h
        assert "b" in h
        assert "B" in h
        assert "c" not in h
        assert h["a"] == "123"
        assert h.get("a") == "123"
        assert h.get("nope", default=None) is None
        assert h.getlist("a") == ["123", "456"]
        assert h.keys() == ["a", "a", "b"]
        assert h.values() == ["123", "456", "789"]
        assert h.items() == [("a", "123"), ("a", "456"), ("b", "789")]
        assert list(h) == ["a", "a", "b"]
        assert dict(h) == {"a": "123", "b": "789"}
        assert repr(h) == "Headers(raw=[(b'a', b'123'), (b'a', b'456'), (b'b', b'789')])"
        assert h == Headers(raw=[(b"a", b"123"), (b"b", b"789"), (b"a", b"456")])
        assert h != [(b"a", b"123"), (b"A", b"456"), (b"b", b"789")]

        h = Headers({"a": "123", "b": "789"})
        assert h["A"] == "123"
        assert h["B"] == "789"
>       assert h.raw == [(b"a", b"123"), (b"b", b"789")]
E       AssertionError: assert [(b'b', b'789...b'a', b'123')] == [(b'a', b'123'...b'b', b'789')]
E         At index 0 diff: (b'b', b'789') != (b'a', b'123')
E         Use -v to get the full diff

tests/test_datastructures.py:145: AssertionError
_____________________________________________________________________ test_queryparams ______________________________________________________________________

    def test_queryparams():
        q = QueryParams("a=123&a=456&b=789")
        assert "a" in q
        assert "A" not in q
        assert "c" not in q
        assert q["a"] == "456"
        assert q.get("a") == "456"
        assert q.get("nope", default=None) is None
        assert q.getlist("a") == ["123", "456"]
>       assert list(q.keys()) == ["a", "b"]
E       AssertionError: assert ['b', 'a'] == ['a', 'b']
E         At index 0 diff: 'b' != 'a'
E         Use -v to get the full diff

tests/test_datastructures.py:182: AssertionError
_______________________________________________________________________ test_formdata _______________________________________________________________________

    def test_formdata():
        upload = io.BytesIO(b"test")
        form = FormData([("a", "123"), ("a", "456"), ("b", upload)])
        assert "a" in form
        assert "A" not in form
        assert "c" not in form
        assert form["a"] == "456"
        assert form.get("a") == "456"
        assert form.get("nope", default=None) is None
        assert form.getlist("a") == ["123", "456"]
>       assert list(form.keys()) == ["a", "b"]
E       AssertionError: assert ['b', 'a'] == ['a', 'b']
E         At index 0 diff: 'b' != 'a'
E         Use -v to get the full diff

tests/test_datastructures.py:215: AssertionError
______________________________________________________________________ test_multidict _______________________________________________________________________

    def test_multidict():
        q = MultiDict([("a", "123"), ("a", "456"), ("b", "789")])
        assert "a" in q
        assert "A" not in q
        assert "c" not in q
        assert q["a"] == "456"
        assert q.get("a") == "456"
        assert q.get("nope", default=None) is None
        assert q.getlist("a") == ["123", "456"]
>       assert list(q.keys()) == ["a", "b"]
E       AssertionError: assert ['b', 'a'] == ['a', 'b']
E         At index 0 diff: 'b' != 'a'
E         Use -v to get the full diff

tests/test_datastructures.py:239: AssertionError

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
em92commented, Jan 26, 2019

I will try ordered dict then

0reactions
tomchristiecommented, Jan 26, 2019

Either we can add sorted to either side of the test comparisons, or you could use an OrderedDict in the datastructures implementations.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Unit tests sometimes failing, sometimes passing - Stack Overflow
Tests can fail intermittently for a number of reasons and identifying why they fail is often revealing about your codebase and environment.
Read more >
"Randomly" failing tests | Apple Developer Forums
We have a test suite that passes when tests are run manually via Xcode. But when run through Xcode Server on about 50...
Read more >
How to Fix Flaky Tests - Semaphore CI
Randomly failing tests are the hardest to debug. Here's a framework you can use to fix them and keep your test suite healthy....
Read more >
Flaky Tests are Not Random Failures | by David Stosik
A flaky test is a test that's unreliable in behaviour, meaning that it yields different results inconsistently. One moment it will pass (and ......
Read more >
Tests that sometimes fail | Hacker News
Unit tests also sometimes fail because of not cleaning up state properly, which only breaks things when tests run in a very specific...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found