CodeQL Not catching known CWE's, not failing build for caught CWE

Got them to run as expected per PR #104

However this pr should definitely cause a failure for:

• CWE-377 py/insecure-temporary-file - via "/opt/hostedtoolcache/CodeQL/0.0.0-20200630/x64/ql/python/ql/src/Security/CWE-377/InsecureTemporaryFile.ql" : { },
• CWE-78 py/command-line-injection - via "/opt/hostedtoolcache/CodeQL/0.0.0-20200630/x64/ql/python/ql/src/Security/CWE-078/CommandInjection.ql" : { },

It did catch CWE-377, but it did not fail the build:

It also didnt show much info. I did not realize it caught this until I accidentally clicked on the changes:

It does show as a check, but only if you happen to expand it. The default view:

If I expand it:

This is very very easy to miss and not at all a good UX. I would think that the action itself would fail the build, or that the check would be easier to spot a failure. At a glance the PR looks mergeable.

That said, it still never caught CWE-78

[myoung34]

I’ll reopen if this pops back up but it seems to be working as expected now

[myoung34]

I’m not convinced it’s a good example though. The “evil” user that can trigger the exec clearly has access to invoke the program from the command line, so from my understanding the “evil” user could just execute the code with python -c “<command>”. (I might be mistaking though)

The example in question is purely to beta test codeql catches. I’m investigating using this at our company, so I was hoping to see how the workflows worked in terms of UI/UX and noticed that CWE-78 didn’t trigger (after my original issues with CWE-377 started working at some point)

Likely i wrote my example too fast and used exec() vs what i really meant: subprocess.call(executable)