Container image scanner started failing today with ApiException: does_not_exist

Describe the bug

We use the Docker image and zap-api-scan.py shipped inside it.

Our automated Zap scanners started failing today (a few hours ago) with exceptions like these:

[ZAP-IO-EventExecutor-3-3] WARN  org.zaproxy.zap.extension.api.API - Bad request to API endpoint [/JSON/ascan/action/setScannerAlertThreshold/] from [127.0.0.1]:
org.zaproxy.zap.extension.api.ApiException: does_not_exist
	at org.zaproxy.zap.extension.ascan.ActiveScanAPI.getScannerFromId(ActiveScanAPI.java:854) ~[zap-D-2022-04-19.jar:D-2022-04-19]
	at org.zaproxy.zap.extension.ascan.ActiveScanAPI.handleApiAction(ActiveScanAPI.java:483) ~[zap-D-2022-04-19.jar:D-2022-04-19]
	at org.zaproxy.zap.extension.api.API.handleApiRequest(API.java:516) ~[zap-D-2022-04-19.jar:D-2022-04-19]
	at org.zaproxy.addon.network.internal.server.http.handlers.ZapApiHandler.handleApiRequest(ZapApiHandler.java:93) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.handlers.ZapApiHandler.handleRequest(ZapApiHandler.java:67) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.handlers.HttpRequestHandler.handleMessage0(HttpRequestHandler.java:32) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.handlers.HttpIncludedMessageHandler.handleMessage(HttpIncludedMessageHandler.java:32) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.MainServerHandler.notifyMessageHandlers(MainServerHandler.java:118) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.MainServerHandler.processMessage(MainServerHandler.java:100) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.LocalServerHandler.processMessage(LocalServerHandler.java:63) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.MainServerHandler.process(MainServerHandler.java:83) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.MainServerHandler.channelRead0(MainServerHandler.java:72) ~[?:?]
	at org.zaproxy.addon.network.internal.server.http.MainServerHandler.channelRead0(MainServerHandler.java:37) ~[?:?]
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) ~[?:?]
	at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
[ZAP-IO-EventExecutor-3-6] INFO  org.parosproxy.paros.core.scanner.Scanner - scanner started
[Thread-6] INFO  org.parosproxy.paros.core.scanner.HostProcess - Scanning 37 node(s) from http://172.17.0.5:6970

And a Python traceback like this:

2022-04-19 16:51:55,052 http://localhost:41691 "GET http://zap/JSON/pscan/view/scanners/ HTTP/1.1" 200 6409
ERROR <class 'AttributeError'>
2022-04-19 16:51:55,054 Unexpected error: <class 'AttributeError'>
Traceback (most recent call last):
  File "/zap/zap-api-scan.py", line 507, in main
    plugin_id = rule.get('id')
AttributeError: 'str' object has no attribute 'get'

I’m not sure if these are related.

Using the old image version (:w2022-04-11) resolves the issue.

I suspect this cropped up somewhere in the past week’s diff: https://github.com/zaproxy/zaproxy/compare/7c032d420e6fb188b052745c938028eb930a0279...d2d0b7155b5bd6a55851c2edd5e13b6e26a930dc Specifically, this PR seems to have touched the lines of code in this stack trace: https://github.com/zaproxy/zaproxy/pull/7206 FYI @ricekot @kingthorin

Steps to reproduce the behavior

1. Run the docker image scanner with the :latest version. We run it like this in CI:

docker run \
    --tty --rm \
    --name=zap-scanner-6728221835413462714 \
    --volume=/tmp/tmp.87XV2BwCDg:/zap/wrk/ owasp/zap2docker-weekly:latest \
    zap-api-scan.py \
        -j -d \
        -t openapi-2344525620.json \
        -f openapi \
        -c zap.conf \
        -r report.html \
        -w report.md \
        -J report.json \
        -z -config ...(rules configuring auth replacements)...

1. Observe the Python traceback and Java ApiException
2. Re-run with last week’s image, observe no issue.

Expected behavior

Expected to see no change in internal image behavior from continuous scans with :latest.

Software versions

:latest

Screenshots

No response

Errors from the zap.log file

Included relevant snippets above. Let me know if you need more logs, I can redact ours as needed.

Additional context

No response

Would you like to help fix this issue?

• Yes

[psiinon]

The weekly and live docker images now have this fix. Thanks for your patience.

[ricekot]

The changes introduced in #7206 result in a proper error message being returned when an incorrect scan rule ID is supplied.

This means that although your scanners were not throwing this error before, they were also not doing what you wanted them to do (they were failing silently).

In this case, it looks like the scan rule ID being supplied to the setScannerAlertThreshold endpoint does not exist. This means that the rule is either not present (e.g. add-on containing it is not installed at the time of the API call), or it may be a passive scan rule ID.