How can I reduce the false-positive mutants for strings?
See original GitHub issueI just ran mutmut
over my toy project mpu and got a lot of false positives for strings.
False Positive String Mutations
Exception messages: They should never be mutated; all of the mutants were false-positives like this:
- raise ValueError("All primes are prime factors of 0.")
+ raise ValueError("XXAll primes are prime factors of 0.XX")
print values: I neither
- print("Number of datapoints: {datapoints}".format(datapoints=len(df)))
+ print("XXNumber of datapoints: {datapoints}XX".format(datapoints=len(df)))
Valuable String Mutations
The following examples could potentially show issues:
comparisons:
- if column_type == "other":
+ if column_type == "XXotherXX":
assignments: I see that assignments are hard to tell. The following could either be in a comparison or in a print (it is a print):
table = [
- ["Column name", "Non-nan", "mean", "std", "min", "25%", "50%", "75%", "max"]
+ ["Column name", "XXNon-nanXX", "mean", "std", "min", "25%", "50%", "75%", "max"]
]
Solutions
Is it possible to configure mutmut to completely disable string mutations or to disable it for those cases?
I have seen Advanced whitelisting and configuraton, but I don’t understand how to apply it in my case.
Issue Analytics
- State:
- Created 3 years ago
- Reactions:1
- Comments:21 (16 by maintainers)
Top Results From Across the Web
Reducing false positive incidental findings with ensemble ...
False positives can be reduced by combining results from orthogonal sequencing methods, but costly. Here we present variant filtering approaches ...
Read more >Reduction of False Positives in Intrusion Detection Based on ...
applied to intrusion detection with promising results for reducing false positives while providing good generalized performance with extremely fast learning ...
Read more >Determining lower limits of detection of digital PCR assays for ...
Digital PCR offers very high assay sensitivity and limit of detection. ... The false positive mutant count is zero for both assays, ...
Read more >TLsub: A transfer learning based enhancement to accurately ...
Although the mutation callers could lower the thresholds, false positives are significantly introduced. The main aim here was to detect the subclonal ...
Read more >What is a false positive and why is having a few around a ...
bar = (String)map38565.get(“keyA-38565”);. As per the OWASP benchmark, this is a false positive that would reduce your score. At ShiftLeft ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@MartinThoma I disagree that you would “never check this because I’ve designed/implemented to not need the test”. The problem with software development is often not how it is implemented now, but how it changes over time. What if someone adds a new exception in future in the
factorize
function and reuses the type with a different error message? You want tests to clearly indicate that a new logical error has been introduced by the change (if indeed that is the case). Yes, you could document somewhere that the exception type is not supposed to be reused, but what if that person forgets, doesn’t notice, is careless, whatever; you still want the tests to clearly indicate that a problem has been introduced. Software development has to account for present and future human cognitive frailty as well correctness in delivering business value.Think of unit/integration tests as the definition of correctness of your software at this moment in time. As the software evolves, the tests should evolve with the software to continue to define correctness. Historically test coverage has been used as a proxy for test quality in absence of other metrics (and it does help), however
mutmut
introduces the possibility of actually testing the quality of the tests.For the exceptions you should also have tests like this (example uses pytest, you can definitely do equivalent using unittest, and presumably other frameworks):
This tests both that the correct type of exception is raised when you expect it, and that the exception raised has the correct message since you might use the same exception multiple times for different error messages.