Queue part of host.json has no impact on how WebJobs perform

Webjobs seem to ignore queue settings in host.json regarding batchSize, newBatchThreshold and visibilityTimeout. The initial goal is to take only one message into execution and the next one when the first is finished. Curently webjob seems to execute as much as possible at once what leads to plenty of Never Finished jobs.

Repro steps

Given we have a valid json file in WebJob folder that describes queues behaviour among others.

"queues": {
    "visibilityTimeout": "00:00:10",
    "batchSize": 6,
    "maxDequeueCount": 5,
    "newBatchThreshold": 3

  }

I create i.e. 40 messages in a for loop, I have long running webjob (ca. 30 seconds when cold start )

Expected behavior

• Having mentioned file I expect to take up to 6 messages from queue in one fetch and take next (next 6 ?) when currently executed job count drops to 3.
• Moreover I expect any message to become visible again after 10s from start of execution as their normall execution times is greater
• Lastly, failed execution should make the message remain in main queue until dequeue count goes to 5 and than finally move it to poison queue

Actual behavior

• Web job takes some large amount of messages in one fetch that has no relation to batchSize : 6 and during execution repeats it making impression that it tries to make it as pararell as possible
• Messages never become visible again. Even Failed and Never finished messages remain invisible although they finally somewhen get again into execution.
• Cannot observe dequeueCount as messages never change to visible again.
• When placing ca. 40 messages of not trival tasks the webjob gets stucked and some of messages go into Never fisnished
• Running locally host.json works well esspecially in correlation with batchSize and newBatchThreshold. I havenot noticed visibilityTimeout to work locally as expected.
• I can observe host.json to influence webjob by setting very low functionTimeout and than having all function gone to failed

Related information

The script uses nodejs QueueTrigger from samples, is deployed to Azure with option 2. WebApp is S1 Standard non-dynamic scaling.

[piorot]

Indeed it works as expected after fixing the issue and the WebHost is now sensitive to knobs changes. @mathewc thank you for committment in the issue.

[mathewc]

I’ve found your issue! Your run.cmd file is incorrect. Rather than using a “start” command like you’ve done which will spawn a child process, you need to simply have .\bin\Microsoft.Azure.WebJobs.Script.Host.exe in the file as the wiki stated. I’ll make that more clear in the wiki.

When you do this, you’ll notice that you no longer have multiple child processes, and instead will have one special process labeled as a WebJob process:

I was able to reproduce the broken behavior with your run.cmd contents, and after fixing that things work as expected.