CloudFormation Error using Certificate in some regions: "ValidationDomain cannot be used with DNS validation"

As of 2020-05-11 I’m seeing a CloudFormation Error “ValidationDomain cannot be used with DNS validation” when trying to create a Certificate using DNS validation.

I have currently deployed stacks that no longer deploy. As far as I can tell this is a change on the CloudFormation side of things. It now breaks in us-west-2 and eu-central-1 but there is no error and things work fine in ap-northeast-1.

Reproduction Steps

My certificate code looks something like this (Scala):

Certificate.Builder.create(scope, "my-ssl-cert")
  .domainName("www.example.com")
  .validationMethod(ValidationMethod.DNS)
  .build()

The resulting CloudFormation template looks like this:

Resources:
  mysslcertF404C81B:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: www.example.com
      DomainValidationOptions:
        - DomainName: www.example.com
          ValidationDomain: example.com
      ValidationMethod: DNS
    Metadata:
      aws:cdk:path: cert-failure-us-west-2/my-ssl-cert/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.38.0,@aws-cdk/assets=1.38.0,@aws-cdk/aws-apigateway=1.38.0,@aws-cdk/aws-applicationautoscaling=1.38.0,@aws-cdk/aws-autoscaling=1.38.0,@aws-cdk/aws-autoscaling-common=1.38.0,@aws-cdk/aws-autoscaling-hooktargets=1.38.0,@aws-cdk/aws-certificatemanager=1.38.0,@aws-cdk/aws-cloudformation=1.38.0,@aws-cdk/aws-cloudfront=1.38.0,@aws-cdk/aws-cloudwatch=1.38.0,@aws-cdk/aws-cognito=1.38.0,@aws-cdk/aws-ec2=1.38.0,@aws-cdk/aws-ecr=1.38.0,@aws-cdk/aws-ecr-assets=1.38.0,@aws-cdk/aws-ecs=1.38.0,@aws-cdk/aws-elasticloadbalancing=1.38.0,@aws-cdk/aws-elasticloadbalancingv2=1.38.0,@aws-cdk/aws-elasticloadbalancingv2-targets=1.38.0,@aws-cdk/aws-events=1.38.0,@aws-cdk/aws-iam=1.38.0,@aws-cdk/aws-kms=1.38.0,@aws-cdk/aws-lambda=1.38.0,@aws-cdk/aws-logs=1.38.0,@aws-cdk/aws-route53=1.38.0,@aws-cdk/aws-route53-targets=1.38.0,@aws-cdk/aws-s3=1.38.0,@aws-cdk/aws-s3-assets=1.38.0,@aws-cdk/aws-sam=1.38.0,@aws-cdk/aws-secretsmanager=1.38.0,@aws-cdk/aws-servicediscovery=1.38.0,@aws-cdk/aws-sns=1.38.0,@aws-cdk/aws-sns-subscriptions=1.38.0,@aws-cdk/aws-sqs=1.38.0,@aws-cdk/aws-ssm=1.38.0,@aws-cdk/cloud-assembly-schema=1.38.0,@aws-cdk/core=1.38.0,@aws-cdk/custom-resources=1.38.0,@aws-cdk/cx-api=1.38.0,@aws-cdk/region-info=1.38.0,jsii-runtime=Java/14

Error Log

I get errors in both us-west-2 and eu-central-1 that look like this:

$ cdk deploy cert-failure-us-west-2
cert-failure-us-west-2: deploying...
cert-failure-us-west-2: creating CloudFormation changeset...
 0/3 | 7:45:10 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata
 0/3 | 7:45:11 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 1/3 | 7:45:11 AM | CREATE_FAILED        | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B) ValidationDomain cannot be used with DNS validation
	new Certificate (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-kernel-jKhSVJ/node_modules/@aws-cdk/aws-certificatemanager/lib/certificate.js:27:22)
	\_ /private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7853:49
	\_ Kernel._wrapSandboxCode (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:8313:20)
	\_ Kernel._create (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7853:26)
	\_ Kernel.create (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7600:21)
	\_ KernelHost.processRequest (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7388:28)
	\_ KernelHost.run (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7328:14)
	\_ Immediate._onImmediate (/private/var/folders/g4/2b_lf4g539z5dy6f3lsf8xfh0000gn/T/jsii-java-runtime11219991876356019748/jsii-runtime.js:7331:37)
	\_ processImmediate (internal/timers.js:456:21)
 2/3 | 7:45:12 AM | CREATE_FAILED        | AWS::CDK::Metadata                   | CDKMetadata Resource creation cancelled
 2/3 | 7:45:13 AM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack           | cert-failure-us-west-2 The following resource(s) failed to create: [mysslcertF404C81B, CDKMetadata]. . Rollback requested by user.
 3/3 | 7:45:17 AM | DELETE_COMPLETE      | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 3/3 | 7:45:17 AM | DELETE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata
 4/3 | 7:45:18 AM | DELETE_COMPLETE      | AWS::CDK::Metadata                   | CDKMetadata
 5/3 | 7:45:19 AM | ROLLBACK_COMPLETE    | AWS::CloudFormation::Stack           | cert-failure-us-west-2

 ❌  cert-failure-us-west-2 failed: Error: The stack named cert-failure-us-west-2 failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE
    at /usr/local/lib/node_modules/aws-cdk/lib/api/util/cloudformation.ts:245:13
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at waitFor (/usr/local/lib/node_modules/aws-cdk/lib/api/util/cloudformation.ts:157:20)
    at Object.deployStack (/usr/local/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:233:26)
    at CdkToolkit.deploy (/usr/local/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:181:24)
    at main (/usr/local/lib/node_modules/aws-cdk/bin/cdk.ts:235:16)
    at initCommandLine (/usr/local/lib/node_modules/aws-cdk/bin/cdk.ts:169:9)
The stack named cert-failure-us-west-2 failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE

However this works fine in ap-northeast-1:

Here is the template:

Resources:
  mysslcertF404C81B:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: www.example.com
      DomainValidationOptions:
        - DomainName: www.example.com
          ValidationDomain: example.com
      ValidationMethod: DNS
    Metadata:
      aws:cdk:path: cert-failure-ap-northeast-1/my-ssl-cert/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.38.0,@aws-cdk/assets=1.38.0,@aws-cdk/aws-apigateway=1.38.0,@aws-cdk/aws-applicationautoscaling=1.38.0,@aws-cdk/aws-autoscaling=1.38.0,@aws-cdk/aws-autoscaling-common=1.38.0,@aws-cdk/aws-autoscaling-hooktargets=1.38.0,@aws-cdk/aws-certificatemanager=1.38.0,@aws-cdk/aws-cloudformation=1.38.0,@aws-cdk/aws-cloudfront=1.38.0,@aws-cdk/aws-cloudwatch=1.38.0,@aws-cdk/aws-cognito=1.38.0,@aws-cdk/aws-ec2=1.38.0,@aws-cdk/aws-ecr=1.38.0,@aws-cdk/aws-ecr-assets=1.38.0,@aws-cdk/aws-ecs=1.38.0,@aws-cdk/aws-elasticloadbalancing=1.38.0,@aws-cdk/aws-elasticloadbalancingv2=1.38.0,@aws-cdk/aws-elasticloadbalancingv2-targets=1.38.0,@aws-cdk/aws-events=1.38.0,@aws-cdk/aws-iam=1.38.0,@aws-cdk/aws-kms=1.38.0,@aws-cdk/aws-lambda=1.38.0,@aws-cdk/aws-logs=1.38.0,@aws-cdk/aws-route53=1.38.0,@aws-cdk/aws-route53-targets=1.38.0,@aws-cdk/aws-s3=1.38.0,@aws-cdk/aws-s3-assets=1.38.0,@aws-cdk/aws-sam=1.38.0,@aws-cdk/aws-secretsmanager=1.38.0,@aws-cdk/aws-servicediscovery=1.38.0,@aws-cdk/aws-sns=1.38.0,@aws-cdk/aws-sns-subscriptions=1.38.0,@aws-cdk/aws-sqs=1.38.0,@aws-cdk/aws-ssm=1.38.0,@aws-cdk/cloud-assembly-schema=1.38.0,@aws-cdk/core=1.38.0,@aws-cdk/custom-resources=1.38.0,@aws-cdk/cx-api=1.38.0,@aws-cdk/region-info=1.38.0,jsii-runtime=Java/14

$ cdk deploy cert-failure-ap-northeast-1
cert-failure-ap-northeast-1: deploying...
cert-failure-ap-northeast-1: creating CloudFormation changeset...
 0/3 | 7:48:46 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata
 0/3 | 7:48:46 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 0/3 | 7:48:47 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B) Resource creation Initiated
 0/3 | 7:48:48 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                   | CDKMetadata Resource creation Initiated
 1/3 | 7:48:48 AM | CREATE_COMPLETE      | AWS::CDK::Metadata                   | CDKMetadata
 1/3 | 7:49:19 AM | CREATE_IN_PROGRESS   | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B) Content of DNS Record is: {Name: _XXXXXXX.www.example.com.,Type: CNAME,Value: _XXXXXXXXXXXXXX.acm-validations.aws.}
1/3 Currently in progress: mysslcertF404C81B
 2/3 | 7:51:20 AM | CREATE_COMPLETE      | AWS::CertificateManager::Certificate | my-ssl-cert (mysslcertF404C81B)
 3/3 | 7:51:22 AM | CREATE_COMPLETE      | AWS::CloudFormation::Stack           | cert-failure-ap-northeast-1

 ✅  cert-failure-ap-northeast-1

Stack ARN:
arn:aws:cloudformation:ap-northeast-1:001234567890:stack/cert-failure-ap-northeast-1/e31b3e34-ffff-ffff-ffff-fffffffffffff

Environment

• CLI Version: 1.38.0 (and 1.32.2)
• Framework Version: 1.38.0 (and 1.32.2)
• OS: Mac
• Language: Scala

Other

I also posted this to the CloudFormation forums: https://forums.aws.amazon.com/thread.jspa?threadID=321860

This is 🐛 Bug Report

[cal5barton]

@citosid the hack is no longer needed. My stacks that were running into problems earlier are working now without any changes.

[MrArnoldPalmer]

An update here.

Cloudformation was experienceing validation errors when performing CloudFormation stack operations for the AWS::CertificateManager resource type. The issue has been resolved and the service is operating normally.

You should be able to remove any escape hatch code for now. Still considering whether we want to remove DomainValidationOptions when ValidationMethod is DNS. Feel free to reopen if this is still occurring for anyone.