[aws-ec2] cfn-init installation hooks

When I add an init section to my EC2 instance that I am booting using an Amazon Linux 2 AMI, the stack fails to deploy because CFN never receives the signal callback from the node.

It appears to me that the root cause is that adding the init section in CDK causes some UserData to be added to make a call to /opt/aws/bin/cfn-init (which is great!), but on this AMI, that script is not installed. So we probably need to do a yum install before the call to cfn-init… I can’t immediately see how to do that, as calls to “addUserData” after the instance object is instantiated result in adding new things to userdata after the cfn-init command.

Reproduction Steps

const instance = new ec2.Instance(this, 'MyInstance', {
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.MICRO),
      machineImage: ec2.MachineImage.lookup({
        name: 'amzn2-ami-ecs-hvm-2.0.20200813-x86_64-ebs',
      }),
      vpc: vpc,
      securityGroup: securityGroup,
      init: ec2.CloudFormationInit.fromElements(
          ec2.InitCommand.shellCommand('sudo yum install tmux'),
      ),
      initOptions: {
        timeout: cdk.Duration.minutes(15)
      }
    })

What did you expect to happen?

Node to boot up and have executed the init commands.

What actually happened?

Stack deploy failed after the cfn init timeout expired.

Environment

• CLI Version : 1.59.0
• Framework Version: 1.59.0
• Node.js Version: 14.7.0
• OS : OSX for cdk env, AL2 for EC2 instance
• Language (Version): TypeScript

Other

---

This is 🐛 Bug Report

[lschierer]

I am struggling to get this to work with default ubuntu AMIs. An example of this in the cdk examples repo would be awesome.

[frankisans]

In addition to Amazon Linux, this problem also occurs with other Linux instances that do not come pre-installed with the CloudFormation helper scripts. If we add a CloudFormation Init configuration to the instance Construct using the init property, it automatically creates a userData that uses cfn-init and cfn-signal scripts, assuming they have been installed on the instance.

#!/bin/bash
# fingerprint: 231cd699be373c15

(
    set +e
    /opt/aws/bin/cfn-init -v --region xxx --stack yyy --resource zzz -c default
    /opt/aws/bin/cfn-signal -e 0 --region xxx --stack yyy --resource zzz
    cat /var/log/cfn-init.log >&2
)

As @cprice404-aws comments, CDK does not allow adding commands to user data to install the cfn tools before using cfn-init, so the provision fails.

As a workaround, I have created an EC2 image builder pipeline to create an AMI with the Cfn helper scripts installed, and use that AMI to create the instance.

I attach the component to install cfn-* scripts:

name: CfnHelperScriptsDocument
description: This is CFN Helper Scripts document.
schemaVersion: 1.0

phases:
  - name: build
    steps:
      - name: CfnHelperScriptsStep
        action: ExecuteBash
        inputs:
          commands:
            - sudo apt update -y
            - sudo apt upgrade -y
            - sudo apt install python3-pip -y
            - pip3 install setuptools
            - sudo mkdir -p /opt/aws/bin
            - sudo wget https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz
            - sudo python3 -m easy_install --script-dir /opt/aws/bin aws-cfn-bootstrap-py3-latest.tar.gz
            - sudo ln -s /root/aws-cfn-bootstrap-py3-latest/init/ubuntu/cfn-hub /etc/init.d/cfn-hub
            - echo "Cfn Helper Scripts! Build."