Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[aws-kinesisanalytics-flink]: Get an error of 'The provided tag map contains duplicate tag key' when updating flink code

See original GitHub issue

First deploy of flink is fine. WHen updating the java code and running a second deploy the stack fails with the error:

The provided tag map contains duplicate tag key

This looks the same as https://github.com/aws/aws-cdk/issues/9652 I’m applying the tag at the root of the app not on the Flink construct itself as i want it on all constructs.

Reproduction Steps

In the CDK App I do this to apply a tag to all constructs

Tags.of(this).add('snowAppId', snowAppId);

In the Flink stack I do this to create a flink instance Stack create flink like this

const flink_app = new flink.Application(this, 'App', {
  applicationName: 'Demo-App-Flink',
  code: flink.ApplicationCode.fromAsset( '...'),
  runtime: flink.Runtime.FLINK_1_11,
  propertyGroups: {
    Connection: { 'bootstrap.servers': props.brokerStrings },
    KafkaSource: { topic: "###" },
    KafkaSink: { topic: "###" },
    Timestream: { database: props.database, table: props.table }
  }
});

What did you expect to happen?

The new code should have been uploaded to S3 and Flink config updated to use it

What actually happened?

Stack failed to deploy - exiting with

  0/5 |7:57:17 PM | UPDATE_FAILED        | AWS::KinesisAnalyticsV2::Application                        | App (AppF1B96344) The provided tag map contains duplicate tag key

Environment

  • **CDK CLI Version :1.111.0
  • **Framework Version:1.111.0
  • **Node.js Version:v14.17.0
  • **OS :Windows 10
  • **Language (Version)😐 TypeScript (3.8.3)

Other

cdk deploy FlinkStack
Including dependency stacks: StorageStack, KafkaStack
[Warning at /KafkaStack/cdk-subnet1] No routeTableId was provided to the subnet 'subnet-06cc77bdc5e83b552'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
[Warning at /KafkaStack/cdk-subnet2] No routeTableId was provided to the subnet 'subnet-082de10fe4d75a3de'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
[Warning at /KafkaStack/cdk-subnet3] No routeTableId was provided to the subnet 'subnet-0f92a2037a9113e7d'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
[Warning at /FlinkStack/cdk-subnet1] No routeTableId was provided to the subnet 'subnet-06cc77bdc5e83b552'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
[Warning at /FlinkStack/cdk-subnet2] No routeTableId was provided to the subnet 'subnet-082de10fe4d75a3de'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
[Warning at /FlinkStack/cdk-subnet3] No routeTableId was provided to the subnet 'subnet-0f92a2037a9113e7d'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
KafkaStack
KafkaStack: deploying...

 ✅  KafkaStack (no changes)

Outputs:
KafkaStack.ExportsOutputFnGetAttclusterBootstrapBrokersBootstrapBrokerStringTls26A4ED35BootstrapBrokerStringTlsF6D16323 = b-2.demo-cluster.v1hggq.c5.kafka.us-west-2.amazonaws.com:9094,b-1.demo-cluster.v1hggq.c5.kafka.us-west-2.amazonaws.com:9094,b-3.demo-cluster.v1hggq.c5.kafka.us-west-2.amazonaws.com
:9094
KafkaStack.ExportsOutputRefcluster611F8AFFA07FC079 = arn:aws:kafka:us-west-2:976206341774:cluster/demo-cluster/3b078248-7b12-48c1-bfec-4bdc1ce96f1f-5
KafkaStack.clusterBootstrapBrokersTls = b-2.demo-cluster.v1hggq.c5.kafka.us-west-2.amazonaws.com:9094,b-1.demo-cluster.v1hggq.c5.kafka.us-west-2.amazonaws.com:9094,b-3.demo-cluster.v1hggq.c5.kafka.us-west-2.amazonaws.com:9094

Stack ARN:
arn:aws:cloudformation:us-west-2:976206341774:stack/KafkaStack/2a508970-dfc3-11eb-b735-020490609721
StorageStack
StorageStack: deploying...

 ✅  StorageStack (no changes)

Outputs:
StorageStack.ExportsOutputReftsdbFC01FFEF = demo-app-db
StorageStack.tableTimestreamTable = demo-app-db|demo-app-table
StorageStack.tsdbTimestreamDatabase = demo-app-db

Stack ARN:
arn:aws:cloudformation:us-west-2:976206341774:stack/StorageStack/290a5100-dfc7-11eb-acf1-021cb123b7ed
FlinkStack
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

IAM Statement Changes
┌───┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬────────┬────────────────────────────────────┬─────────────────┬───────────┐
│   │ Resource                                                                                                                                                                                                                  │ Effect │ Action                             │ Principal       │ Condition │
├───┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼────────┼────────────────────────────────────┼─────────────────┼───────────┤
│ + │ arn:${AWS::Partition}:s3:::${AssetParameters6b200dccb4a5feca18d88999fa3c7457cb54e77fc99bd96413f33d787d19a876S3BucketA5CDD770}                                                                                             │ Allow  │ s3:GetBucket*                      │ AWS:${App/Role} │           │
│   │ arn:${AWS::Partition}:s3:::${AssetParameters6b200dccb4a5feca18d88999fa3c7457cb54e77fc99bd96413f33d787d19a876S3BucketA5CDD770}/*                                                                                           │        │ s3:GetObject*                      │                 │           │
│   │                                                                                                                                                                                                                           │        │ s3:List*                           │                 │           │
├───┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼────────┼────────────────────────────────────┼─────────────────┼───────────┤
│ - │ arn:${AWS::Partition}:s3:::${AssetParametersa9cd009f7a0a03d818ea3a7ea04b474a8c20a93b2fed0c3cc504feb4753597e5S3BucketE2CD97D0}                                                                                             │ Allow  │ s3:GetBucket*                      │ AWS:${App/Role} │           │
│   │ arn:${AWS::Partition}:s3:::${AssetParametersa9cd009f7a0a03d818ea3a7ea04b474a8c20a93b2fed0c3cc504feb4753597e5S3BucketE2CD97D0}/*                                                                                           │        │ s3:GetObject*                      │                 │           │
│   │                                                                                                                                                                                                                           │        │ s3:List*                           │                 │           │
└───┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────┴────────────────────────────────────┴─────────────────┴───────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Do you wish to deploy these changes (y/n)? y
FlinkStack: deploying...
[0%] start: Publishing 6b200dccb4a5feca18d88999fa3c7457cb54e77fc99bd96413f33d787d19a876:current
[50%] success: Published 6b200dccb4a5feca18d88999fa3c7457cb54e77fc99bd96413f33d787d19a876:current
[50%] start: Publishing f3d3a3cc7f26921b237eff24fc5dd7aef8f0465a1f376b8f7918eb3d4b3e8797:current
[100%] success: Published f3d3a3cc7f26921b237eff24fc5dd7aef8f0465a1f376b8f7918eb3d4b3e8797:current
FlinkStack: creating CloudFormation changeset...
  0/5 |7:56:43 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack                                  | FlinkStack User Initiated
  0/5 |7:57:15 PM | UPDATE_IN_PROGRESS   | AWS::KinesisAnalyticsV2::Application                        | App (AppF1B96344)
  0/5 |7:57:17 PM | UPDATE_FAILED        | AWS::KinesisAnalyticsV2::Application                        | App (AppF1B96344) The provided tag map contains duplicate tag key
        new Application (C:\code\cloud\demo-app\infra\node_modules\@aws-cdk\aws-kinesisanalytics-flink\lib\application.ts:167:22)
        \_ new FlinkStack (C:\code\cloud\demo-app\infra\lib\flink-stack.ts:19:23)
        \_ makeApp (C:\code\cloud\demo-app\infra\bin\demo-app.ts:34:17)
        \_ Object.<anonymous> (C:\code\cloud\demo-app\infra\bin\demo-app.ts:63:1)
        \_ Module._compile (internal/modules/cjs/loader.js:1068:30)
        \_ Module.m._compile (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\index.ts:1056:23)
        \_ Module._extensions..js (internal/modules/cjs/loader.js:1097:10)
        \_ Object.require.extensions.<computed> [as .ts] (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\index.ts:1059:12)
        \_ Module.load (internal/modules/cjs/loader.js:933:32)
        \_ Function.Module._load (internal/modules/cjs/loader.js:774:14)
        \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
        \_ main (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\bin.ts:198:14)
        \_ Object.<anonymous> (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\bin.ts:288:3)
        \_ Module._compile (internal/modules/cjs/loader.js:1068:30)
        \_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1097:10)
        \_ Module.load (internal/modules/cjs/loader.js:933:32)
        \_ Function.Module._load (internal/modules/cjs/loader.js:774:14)
        \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
        \_ internal/main/run_main_module.js:17:47
  0/5 |7:57:18 PM | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack                                  | FlinkStack The following resource(s) failed to update: [AppF1B96344].
  2/5 |7:57:41 PM | UPDATE_COMPLETE      | AWS::KinesisAnalyticsV2::Application                        | App (AppF1B96344)
  2/5 |7:57:43 PM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack                                  | FlinkStack
  2/5 |7:57:44 PM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack                                  | FlinkStack

Failed resources:
 7:57:17 PM | UPDATE_FAILED        | AWS::KinesisAnalyticsV2::Application                        | App (AppF1B96344) The provided tag map contains duplicate tag key
        new Application (C:\code\cloud\demo-app\infra\node_modules\@aws-cdk\aws-kinesisanalytics-flink\lib\application.ts:167:22)
        \_ new FlinkStack (C:\code\cloud\demo-app\infra\lib\flink-stack.ts:19:23)
        \_ makeApp (C:\code\cloud\demo-app\infra\bin\demo-app.ts:34:17)
        \_ Object.<anonymous> (C:\code\cloud\demo-app\infra\bin\demo-app.ts:63:1)
        \_ Module._compile (internal/modules/cjs/loader.js:1068:30)
        \_ Module.m._compile (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\index.ts:1056:23)
        \_ Module._extensions..js (internal/modules/cjs/loader.js:1097:10)
        \_ Object.require.extensions.<computed> [as .ts] (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\index.ts:1059:12)
        \_ Module.load (internal/modules/cjs/loader.js:933:32)
        \_ Function.Module._load (internal/modules/cjs/loader.js:774:14)
        \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
        \_ main (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\bin.ts:198:14)
        \_ Object.<anonymous> (C:\code\cloud\demo-app\infra\node_modules\ts-node\src\bin.ts:288:3)
        \_ Module._compile (internal/modules/cjs/loader.js:1068:30)
        \_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1097:10)
        \_ Module.load (internal/modules/cjs/loader.js:933:32)
        \_ Function.Module._load (internal/modules/cjs/loader.js:774:14)
        \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
        \_ internal/main/run_main_module.js:17:47

 ❌  FlinkStack failed: Error: The stack named FlinkStack failed to deploy: UPDATE_ROLLBACK_COMPLETE
    at Object.waitForStackDeploy (C:\Users\nlongton\AppData\Roaming\npm\node_modules\aws-cdk\lib\api\util\cloudformation.ts:307:11)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at Object.deployStack (C:\Users\nlongton\AppData\Roaming\npm\node_modules\aws-cdk\lib\api\deploy-stack.ts:294:26)
    at CdkToolkit.deploy (C:\Users\nlongton\AppData\Roaming\npm\node_modules\aws-cdk\lib\cdk-toolkit.ts:184:24)
    at initCommandLine (C:\Users\nlongton\AppData\Roaming\npm\node_modules\aws-cdk\bin\cdk.ts:213:9)
The stack named FlinkStack failed to deploy: UPDATE_ROLLBACK_COMPLETE


CDK Synth shows this for the flink resource
  AppF1B96344:
    Type: AWS::KinesisAnalyticsV2::Application
    Properties:
      RuntimeEnvironment: FLINK-1_11
      ServiceExecutionRole:
        Fn::GetAtt:
          - AppRole1AF9B530
          - Arn
      ApplicationConfiguration:
        ApplicationCodeConfiguration:
          CodeContent:
            S3ContentLocation:
              BucketARN:
                Fn::Join:
                  - ""
                  - - "arn:"
                    - Ref: AWS::Partition
                    - ":s3:::"
                    - Ref: AssetParameters6b200dccb4a5feca18d88999fa3c7457cb54e77fc99bd96413f33d787d19a876S3BucketA5CDD770
              FileKey:
                Fn::Join:
                  - ""
                  - - Fn::Select:
                        - 0
                        - Fn::Split:
                            - "||"
                            - Ref: AssetParameters6b200dccb4a5feca18d88999fa3c7457cb54e77fc99bd96413f33d787d19a876S3VersionKey965C03DF
                    - Fn::Select:
                        - 1
                        - Fn::Split:
                            - "||"
                            - Ref: AssetParameters6b200dccb4a5feca18d88999fa3c7457cb54e77fc99bd96413f33d787d19a876S3VersionKey965C03DF
          CodeContentType: ZIPFILE
        ApplicationSnapshotConfiguration:
          SnapshotsEnabled: true
        EnvironmentProperties:
          PropertyGroups:
            - PropertyGroupId: Connection
              PropertyMap:
                bootstrap.servers:
                  Fn::ImportValue: KafkaStack:ExportsOutputFnGetAttclusterBootstrapBrokersBootstrapBrokerStringTls26A4ED35BootstrapBrokerStringTlsF6D16323
            - PropertyGroupId: KafkaSource
              PropertyMap:
                topic: cdk-test-topic-1
            - PropertyGroupId: KafkaSink
              PropertyMap:
                topic: foo-out
            - PropertyGroupId: Timestream
              PropertyMap:
                database: demo-app-db
                table: demo-app-table
      ApplicationName: Demo-App-Flink
      Tags:
        - Key: snowAppId
          Value: APM000TEST
    DependsOn:
      - AppRoleDefaultPolicy9CADBAA1
      - AppRole1AF9B530
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: FlinkStack/App/Resource

This is 🐛 Bug Report

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:2
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
gurfinkcommented, Nov 28, 2021

Having same issue with CDK version 1.133.0 Have to now remove inherited tagging - hope this can be fixed soon.

0reactions
shanebishopcommented, Feb 25, 2022

I found a minimal reproducible example (that admittedly uses serverless framework rather than CDK), and I made a demo video to demonstrate how to reproduce this: https://youtu.be/wVH314I052I

I feel rather confident this issue is a CloudFormation issue and not unique to CDK.

Read more comments on GitHub >

github_iconTop Results From Across the Web

InvalidParameterCombination: "Duplicate tag key found in ...
I have the root stack with multiple child stacks(one of which is neptune-application-map.yaml ), and then I have a sub-child stack ( neptune....
Read more >
Create and Run a Kinesis Data Analytics for Flink Application
This section contains the following steps: Create Two Amazon Kinesis Data Streams; Write Sample Records to the Input Stream; Download and Examine the...
Read more >
https://community.cloudera.com/t5/Support-Question...
No information is available for this page.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

DeployCodeBaseCustomResource18AF4C97 failed

Next page

(aws-lambda-nodejs): NodejsFunction support EdgeFunction for Lambda@Edge