Adding delay to some jobs
See original GitHub issueSpark-Bench Version
spark-bench-launch-2.1.1_0.2.2-RELEASE
Spark Version on Your Cluster
Spark.2.2.1
Scala Version on Your Spark Cluster
Scala version 2.11.8 Spark Cluster: Spark Standalone ( 1 master and 2 slaves).
Your Exact Configuration File (with system details anonymized)
spark-bench = {
spark-submit-config = [{
spark-args = {
master = "spark://XX.XX.XX.XX:7077"
}
workload-suites = [
{
descr = "*** KMeans Workload *******"
benchmark-output = "console"
workloads = [
{
name = "kmeans"
input = "/tmp/kmeans-data.csv"
k = 10
}
]
}
]
}]
}
Description of Problem, Any Other Info
I have run the SparkBench using KMeans Workload. Using the Spark dashboard i got the following jobs:
I am studying the Spark performance. I would like to add some delay inside some jobs ( e.g. adding delay inside job1 5 second). Is there any way to add this delay.
Issue Analytics
- State:
- Created 6 years ago
- Comments:7 (4 by maintainers)
Top Results From Across the Web
How to add a delay between processes in a single job?
The only reliable solution I can think of is to use $tries = 1 and to dispatch the same job again if the...
Read more >How to Professionally Delay A Response to a Job Offer
The most tactical way that you can add time to your decision-making period, is to ask specific questions about the offer. Perhaps there...
Read more >Delay Job (DLYJOB) - IBM
The Delay Job (DLYJOB) command causes your current job to wait for a specified number of seconds, or until a specified time of...
Read more >Is it possible to delay a job execution by a set time? - Discussion
As you mention, one approach would be using AAPI to hold the job (run job::hold) and then again to release it (run job::free)...
Read more >4 Simple Steps to Implement “Delayed Job” in Rails - Andolasoft
Here are some of key points to consider: ... Detailed steps to integrate delayed job in a Rails application. Step# 1. Add gem...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I mean that you would need to edit the code inside of Spark itself and recompile Spark. You can use Spark-Bench as-is and just change the
spark-home
parameter to point to your new version of Spark with your custom changes.@Aalnafessah hi~ Did you succeed in adding delay for some spark-jobs? And how?