(custom-resources): AwsCustomResource ignores the configured role if a prior instance was already instantiated

This issue was repurposed following https://github.com/aws/aws-cdk/issues/13601#issuecomment-803574616.

Original report

---

When you enable logging in software.amazon.awscdk.services.elasticsearch, and you are creating log groups, then internally it instantiate the LogGroupResourcePolicy which extends the AWSCustomResource This in turn, sets the AWSCustomResource, which may overwrite any other AWSCustomResource role deployed in the same stack, or have its role overwritten. The reason it overwrites the role, is that the AWSCustomResource role is a singleton for the entire stack

This role will apply to all AwsCustomResource instances in the stack https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_custom-resources.AwsCustomResource.html#role

Reproduction Steps

Python code, Irrelevant code removed for brevity. The main point is to turn on logging with log groups and basically get LogGroupResourcePolicy be to instantiated

# Create my custom resource with a singleton role which I use in the entire stack for all other custom resoruces
policy = AwsCustomResourcePolicy.from_sdk_calls(resources=[f'arn:aws:iot:{region}:{account_id}:rolealias/{role_alias}'])
# This is a singleton class that returns the one role used in my stack
lambda_role_singleton = CustomResourcesLambdaRole(scope)
lambda_role_singleton.add_to_policy(actions=["iam:PassRole"], resources=[role_arn])

AwsCustomResource(scope=self, id='CustomResource', policy=policy, log_retention=log_retention,
                                                on_create=on_create, on_update=on_update, on_delete=on_delete,
                                                resource_type='Custom::AWS-IoT-Role-Alias', role=lambda_role_singleton.role,
                                                timeout=timeout)

# Create the ES domain with logging turned on and log groups
aws_cdk.aws_elasticsearch.Domain(
            scope=self,
            id=id,
            version=ElasticsearchVersion.V7_9,
            logging=self._create_logging_options()
           ....)

def _create_logging_options(self) -> LoggingOptions:
        return LoggingOptions(app_log_enabled=True, slow_index_log_enabled=True, slow_search_log_enabled=True,
                              app_log_group=self._create_log_group(log_group_type=LogGroupType.APP),
                              slow_index_log_group=self._create_log_group(log_group_type=LogGroupType.SLOW_INDEX),
                              slow_search_log_group=self._create_log_group(log_group_type=LogGroupType.SLOW_SEARCH))
 
def _create_log_group(self, log_group_type: str) -> LogGroup:
        return LogGroup(scope=self, id=log_group_type.value.capitalize(), log_group_name=<log-group-name>,
                        removal_policy=RemovalPolicy.DESTROY)

What did you expect to happen?

Both my custom resource and ES domain are deployed

What actually happened?

Deployment failed since I was missing the “iam:PassRole” permissions required to deploy a role alias. Inspecting the cdk.out template, I saw that indeed, this wasn’t present on the role policy. and this was only missing when the ES domain was deployed with logging turned on.

Environment

• CDK CLI Version : 1.91
• Framework Version: 1.91
• Node.js Version: v12.16.0
• OS : MacOS, Linux
• Language (Version): Python 3.8

Other

Unless i’m missing something, a possible solution would allow me to pass the role that will be used or something similar with the logging options.

---

This is 🐛 Bug Report

[iliapolo]

@royby-cyberark Thanks for being so detailed!

Yes you’re right, I neglected to consider the order of creation. Basically what happens is that the role of the custom resource is created when the domain is instantiated.

After this, any role passed to AwsCustomResource is effectively ignored since it reuses the already existing lambda function (which has a different role).

To workaround this, apart from re-arranging the order of instantiation, you can also avoid creating a dedicated role, and simply add policies to the existing one.

my_custom_resource = cr.AwsCustomResource(
            scope=self, id='MyAwsCustomResource', 
            # role=role, # dont pass a role
            policy=cr.AwsCustomResourcePolicy.from_sdk_calls(resources=['*']),
            on_create=cr.AwsSdkCall(
                action='listBuckets',
                service='s3',
                physical_resource_id=cr.PhysicalResourceId.of('BucketsList'),)
        )

my_custom_resource.grant_principal.add_to_policy(...)

I do agree this is confusing though. But it’s more related to how AwsCustomResource behaves, rather than its usage by the elasticsearch domain, which is actually as intended.

Im going to route this to the custom-resources package for further discussion.

FYI @rix0rrr do you think there is something we can do here? Intuitively I would expect that any property that can be passed by the user, should be encoded in the UUID of the function, otherwise they are all similarly ignored.

[royby-cyberark]

@txsutton I wasn’t able to make progress with it. it worked for me so far by allowing ‘*’ which was fine for this use case, so I didn’t get to invest in this further.