question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

False positives when detecting generalized unpacking and bytes formatting

See original GitHub issue

When I try the latest version of vermin (from the master branch, with recent implemenation of generalized unpacking and bytes formatting detection) against some code of my projects, I noticed some false positives when detecting generalized unpacking and bytes formatting.

Case 1:

d = {'a': 'b'}
dict(**d)

Expected: No issues detected (Minimum required versions: ~2, ~3)

Actual: L2: generalized unpacking requires 3.5+

Case 2:

'%x' % 66

Expected: No issues detected (Minimum required versions: ~2, ~3)

Actual: L1: bytes `%` formatting requires 3.5+ (or 2.6+ as `str` synonym)

In this commit, why is isinstance(node.left, ast.Str) necessary? Is it because of some compatibility reasons? b'' is 2.6+, but '%x' % 66 is ~2, ~3.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
gousaiyangcommented, Jan 17, 2020

Looks good. It is fine to close this issue now.

1reaction
gousaiyangcommented, Jan 14, 2020

Here are some test results under Python 3.4. Invalid cases should be regarded as generalized unpacking (they are valid on 3.5+) but valid cases are not.

Unpacking in literals (tuple, list, set, and dictionary displays) is always invalid in Python 3.4: (*(1,),) -> invalid (*(1, 2),) -> invalid (0, *(1, 2)) -> invalid (*(1, 2), 3) -> invalid (0, *(1, 2), 3) -> invalid (*(1, 2), *(3, 4)) -> invalid [*[1]] -> invalid [*[1, 2]] -> invalid [0, *[1, 2]] -> invalid [*[1, 2], 3] -> invalid [0, *[1, 2], 3] -> invalid [*[1, 2], *[3, 4]] -> invalid {*{1}} -> invalid {*{1, 2}} -> invalid {0, *{1, 2}} -> invalid {*{1, 2}, 3} -> invalid {0, *{1, 2}, 3} -> invalid {*{1, 2}, *{3, 4}} -> invalid {**{1: 1}} -> invalid {**{1: 1, 2: 2}} -> invalid {0: 0, **{1: 1, 2: 2}} -> invalid {**{1: 1, 2: 2}, 3: 3} -> invalid {0: 0, **{1: 1, 2: 2}, 3: 3} -> invalid {**{1: 1, 2: 2}, **{3: 3, 4: 4}} -> invalid

In Python 3.4, unpacking in function call parameter list is only allowed at the end of the parameter list, and only one unpacking is allowed. Unpacking in positional argument list and keyword argument list are processed separately: print(*(1,)) -> valid print(*(1, 2)) -> valid print(0, *(1, 2)) -> valid print(*(1, 2), 3) -> invalid print(0, *(1, 2), 3) -> invalid print(*(1, 2), *(3, 4)) -> invalid dict(**{"b": 1}) -> valid dict(**{"b": 1, "c": 2}) -> valid dict(a=0, **{"b": 1, "c": 2}) -> valid dict(**{"b": 1, "c": 2}, d=3) -> invalid dict(a=0, **{"b": 1, "c": 2}, d=3) -> invalid dict(**{"b": 1, "c": 2}, **{"d": 3, "e": 4}) -> invalid foo(0, *(1, 2), a=1, **{"b": 2, "c": 3}) -> valid foo(0, *(1, 2), 3, a=1, **{"b": 2, "c": 3}) -> invalid foo(0, *(1, 2), a=1, **{"b": 2, "c": 3}, d=4) -> invalid

Read more comments on GitHub >

github_iconTop Results From Across the Web

Disallow tuple unpacking for strings · Issue #6406 · python/mypy
I think it would be a good idea to disallow tuple unpacking from ... too many false positives, but this one should be...
Read more >
Detecting Malware with Information Complexity - PMC - NCBI
Our approach classifies disk-resident malware with 97.4% accuracy and a false positive rate of 3%. We demonstrate that its accuracy can be improved...
Read more >
Scalable Platform for Malicious Content Detection Integrating ...
This thesis examines the design, implementation and performance of a scalable analysis plat- form for the detection of malicious content.
Read more >
Transparent and Precise Malware Analysis Using Virtualization
analysis implementations and induce false-positives and false-negatives in an effort to frustrate analysts. This dissertation addresses these problems by ...
Read more >
A Malware and Variant Detection Method Using Function Call ...
The huge influx of malware variants are generated using packing and obfuscating techniques. Current antivirus software use byte signature to ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found