question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[batch] aws_batch.JobDefinition to keep previous revisions Active on update

See original GitHub issue

I tried to look for issues/bugs with this, but I was not able to find any. I am not sure if this is a bug fix or just a feature request. Most of the projects that I work on consist of managing AWS Batch Job Definitions and State Machines to orchestrate large batches of docker containers at once. Each revision of a job definition points to a particular tag of an ECR image for backwards compatibility. In my CDK stack, any time there is any edit to my JobDefinition construct, the stack registers a new job definition revision but deregisters the previous revisions. Is there a way to prevent cdk from deregistering the previous revisions? I can implement this is boto3, but I want to try a native cdk approach.

Reproduction Steps

Initial cdk deploy which creates revision 1 pointing to image:0.0.0

self._job_def = JobDefinition(self, 'JobDef', 
     job_definition_name="job-definition-name",
     container=JobDefinitionContainer(
            image=ecr_image,  # using tag 0.0.0
))

Second cdk deploy which creates revision 2 properly pointing to image:0.0.1, but deregisters revision 1

self._job_def = JobDefinition(self, 'JobDef', 
     job_definition_name="job-definition-name",
     container=JobDefinitionContainer(
            image=ecr_image,  # using tag 0.0.1
))

Error Log

No errors, but there is an unwanted deregister step.

Environment

  • CLI Version : 1.48.0 (build 6080fa8)
  • Framework Version: 1.51.0
  • Node.js Version: node - 12.16.1, npm 6.13.4
  • OS : Mac OS Mojave 10.14.6
  • Language (Version): Python 3.8.3

Other

JobDefinition CF

Above is a link to CloudFormation Resource type page for JobDefinition. The only property I could find for update requires Replacement is job definition name. However, the job definition is always the same in my example.


This is 🐛 Bug Report

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:17 (3 by maintainers)

github_iconTop GitHub Comments

3reactions
sumpforkcommented, Aug 27, 2020

To note, we have a similar issue: We run batch jobs from step functions that can take a long time prior to starting the batch job. When our continuous deployment deploys a new version of the step function, this step function will in many cases point to a new revision of the batch job definition. However, any currently running step function that hasn’t reached the batch job yet will fail because it points to a now inactive job definition revision.

We are going to hack around this by re-activating the last few revisions of the batch job definition after deployment, but that’s obviously less than ideal.

2reactions
jabrennemcommented, Aug 27, 2020

Thank you @iliapolo for looking into this.

I cannot share any code, but your description has helped a lot. I will submit a CloudFormation Coverage Issue to hopefully gain more control over the batch job revisions. I like the idea of a JobDefinitionRevision resource.

For right now, creating a separate batch job definition for each new image:tag will work, and at least it is still a native CDK implementation.

Read more comments on GitHub >

github_iconTop Results From Across the Web

AWS::Batch::JobDefinition - AWS CloudFormation
The AWS::Batch::JobDefinition resource specifies the parameters for an AWS Batch job definition. For more information, see Job Definitions in the AWS Batch ......
Read more >
Get latest job revision while submitting AWS batch job without ...
If name is specified without a revision then the latest active revision is used.
Read more >
Batch — Boto3 Docs 1.26.33 documentation - Amazon AWS
Batch doesn't automatically upgrade the AMIs in a compute environment after ... Compute environments that use Fargate resources must terminate all active ......
Read more >
aws_batch_job_definition | Resources | hashicorp/aws
Browse aws documentation ... Provides a Batch Job Definition resource. ... action - (Required) Specifies the action to take if all of the...
Read more >
Working with AWS Batch Job: Comprehensive Guide to Kickoff ...
6. Creating a Job Definition. As of now, you have created all of the necessary resources to build a pipeline job in AWS...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found