question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

EMR: How do I specify bid price as "OnDemandPrice" in instanceGroups configuration?

See original GitHub issue

Please fill out the sections below to help us address your issue.

What issue did you see ? I am trying to submit a job flow on EMR to start a cluster. I want to set bid price as “OnDemandPrice” instead of USD amount as per the documentation here https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr.html#EMR.Client.run_job_flow

image

However, when I invoke run_job_flow() I get the following exception:

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the RunJobFlow operation: The bid price is invalid. Revise the configuration and resubmit.

My configuration for the instanceGroups looks like this:

"InstanceGroups": [
            {
                "InstanceRole": "MASTER",
                "Market": "SPOT",
                "InstanceType": "r5d.16xlarge",
                "BidPrice": "OnDemandPrice",
                "InstanceCount": 1
            },
            {
                "InstanceRole": "CORE",
                "Market": "SPOT",
                "InstanceType": "r5d.16xlarge",
                "BidPrice": "OnDemandPrice",
                "InstanceCount": 5
            }
        ]

EMR Version: emr-6.2.0

Steps to reproduce If you have a runnable example, please include it as a snippet or link to a repository/gist for larger code examples.

Debug logs Full stack trace by adding boto3.set_stream_logger('') to your code.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:10 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
shanfadniscommented, May 21, 2021

Thank you for the feedback

1reaction
stobrien89commented, May 19, 2021

Hi @shanfadnis,

Thanks for the additional information! I don’t see anything obvious that stands out as problematic, but I’ve reached out to the EMR team to get their thoughts on this. I’ll let you know as soon as I have an update!

Read more comments on GitHub >

github_iconTop Results From Across the Web

InstanceGroupDetail - Amazon EMR - AWS Documentation
BidPrice. If specified, indicates that the instance group uses Spot Instances. This is the maximum price you are willing to pay for Spot...
Read more >
Boto3 API for EMR doesn't support 'OnDemandPrice'
For the time being, if you specify "Market": "SPOT" , BidPriceAsPercentageOfOnDemandPrice will default to 100%, aka the on-demand price.
Read more >
EMR — Boto3 Docs 1.26.33 documentation - AWS
Specify OnDemandPrice to set the amount equal to the On-Demand price, or specify an amount in USD. InstanceType (string) -- [REQUIRED]. The EC2...
Read more >
aws_emr_cluster | Resources | hashicorp/aws
To configure Instance Groups for task nodes, ... (Optional) Bid price, as a percentage of On-Demand price, for each EC2 Spot instance as...
Read more >
aws emr add-instance-groups - Fig.io
Specify OnDemandPrice to set the amount equal to the On-Demand price, or specify an amount in USD.[EbsConfiguration] - Specifies additional Amazon EBS ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found