question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Excessive identical job parallelism in workflow

See original GitHub issue

RedEye folk,

Could you take a look at your testing workflow. When it kicks off it is occupying over half of our organization’s available runners, and has starved out some of the other projects.

  • Please document the purpose of 12 identical jobs running in parallel in the workflow. e.g., the parallel tests are non-deterministic.
  • Please consider using max-parallel to limit the matrix to a couple of jobs at a time if this parallelism isn’t necessary.

Thank, @felddy

https://github.com/cisagov/RedEye/blob/336919c97d620f40aaf403131ee922b7b169221d/.github/workflows/test.yml#L47-L51

https://github.com/cisagov/RedEye/blob/336919c97d620f40aaf403131ee922b7b169221d/.github/workflows/test.yml#L90-L94

Issue Analytics

  • State:closed
  • Created 10 months ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

4reactions
sang2925commented, Nov 15, 2022

Sorry about this. I will reduce the amount of runners that the cypress tests are using.

2reactions
jsf9kcommented, Nov 17, 2022

Sorry about this. I will reduce the amount of runners that the cypress tests are using.

Please don’t forget to specify the purpose of running multiple, identical copies of the same job. (The value of containers from the matrix strategy is not used as far as I can tell, which is why I am saying that all the jobs are identical.)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Run jobs in parallel with workflows to decrease your build times
If jobs are a collection of steps, then workflows can be thought of as a collection of jobs. Similar to our example of...
Read more >
Parallel executions with Google Workflows - Medium
You can't parallelize different callable workflow, it's always the same with custom parameters. In reality, it's no longer a limitation, see the ...
Read more >
Ligo is a typical embarrassingly parallel workflow. Tasks that ...
Tasks that execute the same program on different data are placed at the ... For instance, assigning too much memory, although resulting in...
Read more >
Ability to run tasks in parallel - Visual Studio Feedback
There are often times when you want to run two or more tasks in parallel and then move on when all are complete....
Read more >
Build Pipelines: efficient CI/CD workflows with parallelization
In Build Pipelines is that artifact sharing becomes easier: if you build your app once for testing, you can run any number of...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found