(stepfunctions-tasks): AWS Batch integration with container overrides breaks with token values

What is the problem?

I was eagerly awaiting 1.149 as it appeared to fix the issue that prevents container overrides of memory and cpu:

https://github.com/aws/aws-cdk/issues/18993 https://github.com/aws/aws-cdk/pull/19298

However, now I get the following error:

{
  "resourceType": "batch",
  "resource": "submitJob.sync",
  "error": "Batch.ClientException",
  "cause": "Value -1.888154589709072e+289 for type MEMORY in resourceRequirement is not valid. Provide a valid number as input. (Service: AWSBatch; Status Code: 400; Error Code: ClientException; Request ID: dcd8c27a-baac-414a-9b64-da5733cc1344; Proxy: null)"
}

Reproduction Steps

This is the code that sets up the container overrides

BatchContainerOverrides containerOverrides = BatchContainerOverrides.builder()
                .command(JsonPath.listAt("$." + SOLVER_TASK_PARAMETERS + "." + COMMAND))
                .vcpus(JsonPath.numberAt("$." + SOLVER_TASK_PARAMETERS + "." + CPU))
                .memory(Size.mebibytes(JsonPath.numberAt("$." + SOLVER_TASK_PARAMETERS + "." + MEMORY)))
                .build();

After I updated to 1.149 there is still no resourceRequirements field on BatchContainerOverrides. Based on the fix it appears that these values would just be copied from the top-level vcpus and memory.

Previously I was receiving the warning that my container overrides were being ignored, but the job would still execute.

What did you expect to happen?

I expected that with this update my container overrides would be respected.

What actually happened?

{
  "resourceType": "batch",
  "resource": "submitJob.sync",
  "error": "Batch.ClientException",
  "cause": "Value -1.888154589709072e+289 for type MEMORY in resourceRequirement is not valid. Provide a valid number as input. (Service: AWSBatch; Status Code: 400; Error Code: ClientException; Request ID: dcd8c27a-baac-414a-9b64-da5733cc1344; Proxy: null)"
}

CDK CLI Version

1.149.0

Framework Version

No response

Node.js Version

12.22.11

OS

MacOS 10.15.7

Language

Java

Language Version

Java 11

Other information

No response

[peterwoodworth]

I think I see what’s happening here.

Because you’re using JsonPath.numberAt(), that will return a token. We’re trying to call the direct value of a token, which won’t work if we want JsonPath.

What’s interesting is that the same applies for GpuCount https://github.com/aws/aws-cdk/blob/8c60d4d250cdc2f796c3cfc69c517b17804b6463/packages/%40aws-cdk/aws-stepfunctions-tasks/lib/batch/submit-job.ts#L299, just no one has run into this error before with that?

We need to consider the case where the value is a token

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.