question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Cloudformation script fails on creating lambda due to missing IAM role

See original GitHub issue

Describe the bug We (@npalm and @joostwestra have created the MLOps framwork via the stack published on AWS (ttps://solutions-reference.s3.amazonaws.com/aws-mlops-framework/latest/aws-mlops-single-account-framework.template). All worked fine till today. As of today we got an error during creation of the stack. Creating a lamba fails with an error that the required role not exists. However the specific role was created before. See log below.

To Reproduce

  1. Create cloud formation stack based on: ttps://solutions-reference.s3.amazonaws.com/aws-mlops-framework/latest/aws-mlops-single-account-framework.template (used region eu-west-1)
  2. Just wait, after a few mintues the stack will rollback.

Expected behavior Should succeed (as before)

Please complete the following information about the solution:

  • latest

To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, “(SO0136) - AWS MLOps Framework. Version v1.0.0”.

  • Region: eu-west-1
  • Were there any errors in the CloudWatch Logs?

Screenshots

Additional context Log from cloudformation:



2021-08-26 16:41:39 UTC+0200 | niek2-mlops | ROLLBACK_IN_PROGRESS | The following resource(s) failed to create: [accessLogs751B1465, PipelineOrchestrationLambdaRestApiAccountB62D7499, SolutionHelperC369AAA8]. Rollback requested by user.
-- | -- | -- | --
2021-08-26 16:41:38 UTC+0200 | PipelineOrchestrationLambdaRestApiAccountB62D7499 | CREATE_FAILED | Template error: IAM role niek2-mlops-PipelineOrchestrationLambdaRestApiClou-L6QKNJREWOOG doesn't exist
2021-08-26 16:41:37 UTC+0200 | accessLogs751B1465 | CREATE_FAILED | Resource creation cancelled
2021-08-26 16:41:37 UTC+0200 | SolutionHelperC369AAA8 | CREATE_FAILED | Template error: IAM role niek2-mlops-SolutionHelperServiceRoleB95702BC-QBXJKZ5M45T3 doesn't exist
2021-08-26 16:41:36 UTC+0200 | PipelineOrchestrationLambdaFunctionServiceRoleE64D6F74 | CREATE_COMPLETE | -
2021-08-26 16:41:36 UTC+0200 | PipelineOrchestrationLambdaRestApiCloudWatchRole6F4B9723 | CREATE_COMPLETE | -
2021-08-26 16:41:35 UTC+0200 | mlopscloudformationrole867810B0 | CREATE_COMPLETE | -
2021-08-26 16:41:35 UTC+0200 | CustomResourceLambdaServiceRole757E6AFA | CREATE_COMPLETE | -
2021-08-26 16:41:34 UTC+0200 | SolutionHelperServiceRoleB95702BC | CREATE_COMPLETE

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
tabdunabicommented, Aug 26, 2021

Hi @npalm, Thank you for reporting the issue. I was able to reproduce the issue, and we are investigating the root cause. I’ll get back to you ASAP.

1reaction
npalmcommented, Aug 26, 2021

Thanks for the quick response

Read more comments on GitHub >

github_iconTop Results From Across the Web

Resolve the role_arn error when updating or deleting an AWS ...
1. Open the IAM console. 2. In the navigation pane, choose Roles. 3. In the Role name column, choose the IAM role that's...
Read more >
AWS CloudFormation - resources failed to create error
There is a default IAM role "aws-opsworks-service-role IAM role" in my account which is created when we first create an OpsWorks stack.
Read more >
10 Solutions to Common CloudFormation Errors - Medium
When creating a CloudFormation that includes Lambda Function resources, you should check whether those Lambdas have Environmental Variables. If ...
Read more >
Top 10 Serverless Deployment Errors (and How to Fix Them)
The state returned from AWS CloudFormation is usually UPDATE_ROLLBACK_FAILED or DELETE_FAILED , and the error will look something like this:.
Read more >
Template error: IAM role stack-Lambda-XXXXX doesn't exist ...
Create, Delete, List, Get, and Update API actions may be impacted in multiple regions. Other AWS services like AWS CloudFormation and AWS Lambda...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found