question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Parallelism within the package tree

See original GitHub issue

It’s not clear how lerna parallelism works in regards to package dependency and the build order. For example, if I have the follow dependency tree (via devDependencies/dependencies):

packages/
  +--package1
  +   +--package3
  +   +--package2
  +--package2
  +   +--package4
  +--package3
  +   +--package4
  +--package4

When I run lerna run build, obviously package4 is built first, and package1 is built last, but then it’s not clear if package2 and package4 are build in parallel, or one after the other (even though they have no relation to each other.

Expected Behavior

Packages should be built in parallel as much as possible.

Current Behavior

Not sure. The docs aren’t clear on this.

Context

  • How has this issue affected you? What are you trying to accomplish?
    • Faster build times

Your Environment

Executable Version
lerna --version 3.11.1
npm --version 6.7.0
node --version 10.12.0
OS Version
macOS Mohave 10.14.2

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
evocateurcommented, Feb 18, 2019

Not directly, though both lerna exec and lerna run will execute their arguments with maximal topological concurrency.

Programmatically, all the bits are available as separate packages, so it wouldn’t be too hard to retrieve the same topologically batched package list that Lerna uses internally:

const { getPackages } = require('@lerna/project');
const batchPackages = require('@lerna/batch-packages');

(async () {
  const cwd = process.cwd();
  const pkgs = await getPackages(cwd);
  const batches = batchPackages(pkgs);

  for (const batch of batches) {
    // all packages in a batch are safe to operate on concurrently
    for (const pkg of batch) {
      console.log(pkg.name);
      console.log(pkg.location); // absolute path to leaf directory
    }
  }
}());
0reactions
evocateurcommented, Feb 27, 2019

@wbern That’s a separate issue, which I believe already exists (possibly in triplicate).

Read more comments on GitHub >

github_iconTop Results From Across the Web

[ANN] FileTrees.jl -- easy everyday parallelism on trees of files
It is a thin layer of abstraction that helps solve common problems in working with multiple files of data. It also abstracts away...
Read more >
Parallel Processing in R
Parallel processing (in the extreme) means that all the f# processes start simultaneously and run to completion on their own. If we have...
Read more >
Overview of Parallel Processing in R - Learn by Marketing
How it Works: The gbm package relies on the built-in parallel package. The search for the best decision tree is done across the...
Read more >
lidR-parallelism: Parallel computation in lidR - Rdrr.io
This document explains how to process point clouds taking advantage of parallel processing in the lidR package. The lidR package has two ...
Read more >
Optimizations and parallel processing - tidymodels/tune
For example, suppose a boosted tree is fit with 1000 trees. ... In order to know what models allow this, the parsnip package...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found