DDB Stream Lambda creation fails due to IAM policy not being updated first

This is a Bug Report

Description

• What went wrong?

While updating an existing deployment with a new DDB Stream lambda, while simultaneously updating the IAM policy for the role and the DDB table to add the stream, the AWS::Lambda::EventSourceMapping fails to create with the following error:

Cannot access stream arn:aws:dynamodb:us-west-2:123456789012:table/logger-api-gateway-stage-2/stream/2017-07-26T18:04:04.509. Please ensure the role can perform the GetRecords, GetShardIterator, DescribeStream, and ListStreams Actions on your stream in IAM.

These permissions are due to be added in the update, but because the AWS::Lambda::EventSourceMapping is not set to depend on the IAM role, the updates don’t happen in the correct sequence.

• What did you expect should have happened?

AWS::Lambda::EventSourceMapping’s that are for DynamoDB Streams should have “DependsOn”: “IamRoleLambdaExecution”

• What was the config you used?

  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:DescribeStream
        - dynamodb:GetItem
        - dynamodb:GetRecords
        - dynamodb:GetShardIterator
        - dynamodb:ListStreams
        - dynamodb:PutItem
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:UpdateItem
      Resource:
        - Fn::GetAtt:
          - LogsTable
          - StreamArn
functions:
  bus:
    description: Processes changes from DynamoDB and relays as appropriate
    handler: logs/bus.process
    events:
      - stream:
          type: dynamodb
          arn:
            Fn::GetAtt:
              - LogsTable
              - StreamArn

• What stacktrace or error message from your provider did you see?

  Serverless Error ---------------------------------------
 
  An error occurred while provisioning your stack: BusEventSourceMappingDynamodbLogsTable - Cannot access stream arn:aws:dynamodb:us-west-2:012345678901:table/logger-api-gateway-stage-2/stream/2017-07-26T18:04:04.509. Please ensure the role can perform the GetRecords, GetShardIterator, DescribeStream, and ListStreams Actions on your stream in IAM..

Additional Data

• Serverless Framework Version you’re using:

1.16.1

• Operating System:

ubuntu 16.04

[alexcasalboni]

Hi @flyinbutrs, I am experiencing the same problem with Kinesis Streams.

I get a similar error:

An error occurred: MyFunctionEventSourceMappingMyKinesisStream - Cannot access stream arn:aws:kinesis:us-east-1:XXX:stream/YYY. Please ensure the role can perform the GetRecords, GetShardIterator, DescribeStream, and ListStreams Actions on your stream in IAM.

Have you tested your suggested fix related to dependsOn?

CC @pmuens you can reproduce the bug by starting with a yaml configuration with no iamRoleStatements and no events, such as:

service: xxx
provider:
  name: aws
  runtime: python2.7
functions:
  myFunction:
    handler: handler.my_handler

You deploy it, and then you update the config to:

service: xxx
provider:
  name: aws
  runtime: python2.7
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - kinesis:GetRecords
        - kinesis:GetShardIterator
        - kinesis:DescribeStream
        - kinesis:ListStreams
      Resource:
        - Fn::GetAtt:
          - MyKinesisStream
          - Arn

functions:
  myFunction:
    handler: handler.my_handler
    events:
      - stream:
          type: kinesis
          batchSize: 100
          arn:
            Fn::GetAtt:
              - MyKinesisStream
              - Arn

resources:
  Resources:
    MyKinesisStream:
      Type: AWS::Kinesis::Stream
      Properties:
        ShardCount: 1

I believe this did not happen until a few months ago because AWS was not validating Lambda’s permissions when you created a new EventSourceMapping.

I wouldn’t consider the bug super critical, as everything works fine if you sls remove and then sls deploy again, but it can be quite frustrating for beginners.

[jheising]

Ran into the same problem. Not sure if this helps, but I think this is actually a problem with the error message— it doesn’t list all the permissions you actually need. I found at https://docs.amazonaws.cn/en_us/lambda/latest/dg/with-kinesis.html under “Execution role permissions” it says you need more permissions. I’ve updated my serverless.yml with the following and it seems to work:

      - Effect: "Allow"
        Action:
          - kinesis:DescribeStream
          - kinesis:DescribeStreamSummary
          - kinesis:GetRecords
          - kinesis:GetShardIterator
          - kinesis:SubscribeToShard
        Resource: your_stream_arn

      - Effect: "Allow"
        Action:
          - kinesis:ListStreams
          - kinesis:ListShards
        Resource: "*"