Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Adding delay to some jobs

See original GitHub issue

Spark-Bench Version


Spark Version on Your Cluster


Scala Version on Your Spark Cluster

Scala version 2.11.8 Spark Cluster: Spark Standalone ( 1 master and 2 slaves).

Your Exact Configuration File (with system details anonymized)

spark-bench = {
  spark-submit-config = [{
    spark-args = {
      master = "spark://XX.XX.XX.XX:7077"
    workload-suites = [
      descr = "***  KMeans Workload *******"
      benchmark-output = "console"
      workloads = [

         name = "kmeans"
         input = "/tmp/kmeans-data.csv"
         k = 10

Description of Problem, Any Other Info

I have run the SparkBench using KMeans Workload. Using the Spark dashboard i got the following jobs:
screen shot 2018-01-09 at 4 54 08 pm

I am studying the Spark performance. I would like to add some delay inside some jobs ( e.g. adding delay inside job1 5 second). Is there any way to add this delay.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

ecurtincommented, Jan 19, 2018

I mean that you would need to edit the code inside of Spark itself and recompile Spark. You can use Spark-Bench as-is and just change the spark-home parameter to point to your new version of Spark with your custom changes.

xiandong79commented, Apr 5, 2018

@Aalnafessah hi~ Did you succeed in adding delay for some spark-jobs? And how?

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to add a delay between processes in a single job?
The only reliable solution I can think of is to use $tries = 1 and to dispatch the same job again if the...
Read more >
How to Professionally Delay A Response to a Job Offer
The most tactical way that you can add time to your decision-making period, is to ask specific questions about the offer. Perhaps there...
Read more >
Delay Job (DLYJOB) - IBM
The Delay Job (DLYJOB) command causes your current job to wait for a specified number of seconds, or until a specified time of...
Read more >
Is it possible to delay a job execution by a set time? - Discussion
As you mention, one approach would be using AAPI to hold the job (run job::hold) and then again to release it (run job::free)...
Read more >
4 Simple Steps to Implement “Delayed Job” in Rails - Andolasoft
Here are some of key points to consider: ... Detailed steps to integrate delayed job in a Rails application. Step# 1. Add gem...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found