question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

(pipelines): support caching in simple synth action

See original GitHub issue

Allow for caching in SimpleSynthAction

Use Case

Allowing for caching in the SimpleSynthAction makes it easy to decrease build times - e.g. caching incremental typescript lambda builds, caching npm packages, or even re-use unchanged resources - which can become quite incredible when using CDK Pipelines (like when a re-build is triggered after a pipeline self-mutate).

Proposed Solution

This would simply add a couple additional optional fields (cache and cachePaths) onto the SimpleSynthAction static methods, which could be used like:

SimpleSynthAction.standardNpmSynth({
  sourceArtifact,
  cloudAssemblyArtifact,
  installCommand: 'npm ci --cache .npm',
  buildCommand: "npm run ci-build",
  cachePaths: ['dist/**/*', '.npm/**/*', './.git/objects/**/*', '.git-hash'],
  cache: Cache.bucket(new Bucket(this, 'Cache'))
});

Currently, you can either create the whole build configuration yourself or copy and modify the source of the SimpleSynthAction, but these both seem untenable considering how easy it is to pass these options to the constructed internals.

Other

These parameters just get passed along into the SimpleSynthAction without any direct transformation, so this is really simple:

const buildSpec = BuildSpec.fromObject({
  ....
  cache: {
    paths: this.props.cachePaths
  }
});

and

const project = new PipelineProject(scope, 'CdkBuildProject', {
  ...
  cache: this.props.cache
});

SimpleSynthAction docs SimpleSynthAction source CodeBuild caching

  • 👋 I may be able to implement this feature request
  • ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:27
  • Comments:8 (3 by maintainers)

github_iconTop GitHub Comments

3reactions
Pythoner6commented, Apr 2, 2021

I’ve got an application where this would be really helpful - it has a bunch of rust-based dockerized lambdas, which building from scratch can take 10 minutes, and most of that time is compiling all the dependencies from source. It would be really nice to be able to have those bits cached to cut out a substantial portion of that build time.

1reaction
simlevesquecommented, Sep 3, 2021

With the new API you can provide a partialBuildSpec to configure caching. The SimpleSynthAction is now part of the legacy API.

@hoegertn

Yes, I use a partialBuildSpec which configures caching for some folders. But there is no persistence without a PipelineProject cache like the OP is saying. Right now there are no way to share a yarn/npm cache between different pipelines invocation if I’m not mistaken.

Read more comments on GitHub >

github_iconTop Results From Across the Web

class SimpleSynthAction · AWS CDK
Create a standard NPM synth action. Uses npm ci to install dependencies and npx cdk synth to synthesize. If you need a build...
Read more >
Caching in GitLab CI/CD
A cache is one or more files a job downloads and saves. Subsequent jobs that use the same cache don't have to download...
Read more >
Pipeline caching - Azure - Microsoft Learn
Pipeline caching can help reduce build time by allowing the outputs or downloaded dependencies from one run to be reused in later runs, ......
Read more >
AWS CDK Pipelines: Real-World Tips and Tricks (Part 2)
json is used for local development and for predefined pipeline actions such SimpleSynthAction.standard_npm_synth() . If no standard standard synth action is ...
Read more >
PSO Caching - Unreal Engine Documentation
The Pipeline State Object (PSO) caching tools help reduce the time it takes to reload an Unreal Engine 4 (UE4) level and it's...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found