question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Make --strict imply --strict-config / xfail_strict?

See original GitHub issue

With pytest 4.5.0 (May 2019), we introduced --strict-markers as a replacement for --strict, and said:

The existing --strict option has the same behavior currently, but can be augmented in the future for additional checks.

With #7286, now we also have --strict-config. Yet, --strict is still an alias for --strict-markers.

Given that pytest 6 is allowed to do breaking changes, wouldn’t it make sense for --strict to imply both --strict-markers and --strict-config now?

(Though, on a second thought: Do we actually need --strict-markers in pytest 6 still, given the warning system? Probably still useful as a convenient way to turn those warnings into errors?)

This would also make pytest more in line with other tools with a --strict flag - mypy and tsc (the typescript compiler) come to mind, where --strict implies various different strict flags.

cc @gnikonorov (who added --strict-config), @webknjaz and @DahlitzFlorian (some existing discussion in #7233), and @nicoddemus (#5023) - what do you think?

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:36 (36 by maintainers)

github_iconTop GitHub Comments

4reactions
nicoddemuscommented, Jul 22, 2020

In that case I think we should just drop the --strict alias in v6.0 to make it semver-available as a comprehensive flag for all 3 modes in V6.1

Indeed I believe this is the safest option: the breaking change in 6.0 then becomes the fact that --strict gets removed, and then we can reintroduce it in a minor release (6.1 for example). This seems better to me than changing the behavior of an existing option, because a missing option is a hard error, the former is a change in behavior which might be harder to track down if one does not read the CHANGELOG. Of course it might happen that an user skips directly from 5.4 to 6.1 and gets the behavior change directly, but still seems like it is less likely to cause problems.

As for if it should also cover xfail_strict, I think for the sake of avoiding confusion, --strict should also cover xfail_strict, or for consistency we don’t reintroduce --strict as a command-line option but as a config option instead.

3reactions
DahlitzFloriancommented, Jul 15, 2020

Totally agree with @The-Compiler. In fact, --strict should be a shortcut for using all --strict* options.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to use skip and xfail to deal with tests that cannot succeed
An xfail means that you expect a test to fail for some reason. ... You can change this by setting the strict keyword-only...
Read more >
How and why I use pytest's xfail - Paul Ganssle
Still, strict xfailing tests is better than the alternative, which gives you no insight into whether or not your test was failing. Reasons...
Read more >
marking a test as xfail with double parameterize in pytest
You can't mark the test based on the complete set of arguments in pytest.mark.parametrize / pytest.param , the information about other ...
Read more >
pytest: xfail vs xpass and all test statuses (beginner - YouTube
today I talk about all the different types of statuses that a pytest test can have! I also explain my rationale for using...
Read more >
171 - How and why I use pytest's xfail - Paul Ganssle | pythontest
The Git integration makes all of my Git workflows super easy, ... I mean, I think people don't like the idea of X...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found