(core): `CustomResourceProvider` writes to `node_modules`

What is the problem?

On instantiation, the CustomResourceProvider class writes a file into the node_modules directory instead of creating a safe temporary directory to stage local assets. What’s more, for users of Yarn berry with PnP, node_modules is a read-only ZipFS filesystem, which raises an exception when the write is attempted.

Reproduction Steps

const bucket = new s3.Bucket(this, 'MyBucket', {
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  autoDeleteObjects: true, // <-- this line causes synth to fail, uses CustomResourceProvider
});

What did you expect to happen?

A stack would be synthesized containing a custom resource to delete objects from the above bucket in the event the bucket is ever removed from the stack.

What actually happened?

me@host % yarn cdk synthesize
/Users/me/workspace/cdk/.pnp.cjs:17448
  return Object.assign(new Error(`${code}: ${message}`), {
                       ^
Error: EROFS: read-only filesystem, open '/node_modules/@aws-cdk/aws-s3/lib/auto-delete-objects-handler/__entrypoint__.js'
    at makeError (/Users/me/workspace/cdk/.pnp.cjs:17448:24)
    at EROFS (/Users/me/workspace/cdk/.pnp.cjs:17478:10)
    at ZipFS.prepareWriteFile (/Users/me/workspace/cdk/.pnp.cjs:19787:30)
    at ZipFS.writeFileSync (/Users/me/workspace/cdk/.pnp.cjs:19775:14)
    at fallback (/Users/me/workspace/cdk/.pnp.cjs:20617:14)
    at /Users/me/workspace/cdk/.pnp.cjs:20637:18
    at /Users/me/workspace/cdk/.pnp.cjs:20951:58
    at ZipOpenFS.getZipSync (/Users/me/workspace/cdk/.pnp.cjs:21096:14)
    at ZipOpenFS.makeCallSync (/Users/me/workspace/cdk/.pnp.cjs:20951:17)
    at /Users/me/workspace/cdk/.pnp.cjs:20631:19 {
  code: 'EROFS'
}
Subprocess exited with error 1

CDK CLI Version

1.131.0 (build 7560c79)

Framework Version

1.131.0

Node.js Version

v16.3.0

OS

macOS Big Sur 11.6.1

Language

Typescript

Language Version

TypeScript (4.4.4)

Other information

Previously reported as a problem in the aws-s3 module, closed by that module’s maintainer:

https://github.com/aws/aws-cdk/issues/16552

Most likely source of the problem:

https://github.com/aws/aws-cdk/blob/5aa6ac0008b2bd0556fda021124f6838173274b4/packages/%40aws-cdk/core/lib/custom-resource-provider/custom-resource-provider.ts#L165-L166

A temporary directory should be created (the core module provides FileSystem.mkdtemp) and the code should be written to that instead of assuming that the module directory can accept writes. Specifics for Yarn 2.x can be viewed at:

https://yarnpkg.com/features/pnp

[izaakschroeder]

I lied. Referencing the original version above only passes node’s cursory checks and doesn’t actually build properly. The files need to be copied. A better version follows (and appears to work):

import {CustomResourceProvider, CustomResourceProviderProps} from 'aws-cdk-lib';
import {Construct} from 'constructs';
import * as path from 'path';
import * as os from 'os';
import * as fs from 'fs';
import * as crypto from 'crypto';

const copyFilesSync = (srcDir: string, dstDir: string) => {
  fs.readdirSync(srcDir).forEach((file) => {
    const src = srcDir + '/' + file;
    const dst = dstDir + '/' + file;
    var stat = fs.statSync(src);
    if (stat && stat.isDirectory()) {
      fs.mkdirSync(dst);
      copyFilesSync(src, dst);
    } else {
      fs.writeFileSync(dst, fs.readFileSync(src));
    }
  });
};

const tmp = os.tmpdir();
const getOrCreateProvider = CustomResourceProvider.getOrCreateProvider;
CustomResourceProvider.getOrCreateProvider = (
  scope: Construct,
  id: string,
  props: CustomResourceProviderProps,
) => {
  const hash = crypto
    .createHash('sha1')
    .update(props.codeDirectory)
    .digest('hex');
  const codeDirectory = path.join(tmp, `cdk-crp-${hash}`);
  if (!fs.existsSync(codeDirectory)) {
    fs.mkdirSync(codeDirectory);
    copyFilesSync(props.codeDirectory, codeDirectory);
  }
  return getOrCreateProvider(scope, id, {
    ...props,
    codeDirectory,
  });
};

[rix0rrr]

I agree this is poor behavior.