Replace SDB with S3 in AWS job store

Quoting an AWS support professional in case 1767267511:

I would recommend seeing if you would consider DynamoDB to replace your SimpleDB solution. DynamoDB is essentially the successor of SimpleDB, which is slowly being pulled out from active development. In fact, we’re no longer offering that to new customers at this point.

If Toil should run on newly opened AWS accounts, we need to phase out SimpleDB.

I propose that we create a new, second implementation of the AWS job store that uses DynamoDB. The new implementation should be accessible under the aws job store locator, while the old one becomes aws_old.

The reason I didn’t use DynamoDB in the first place was the payment model, which is based on a flat rate as a function of a configurable (“provisioned” in Amazon lingo) request volume. Toil would have to set that request volume to user-specified value (with a sensible default) before a workflow starts and make sure that it configures it back to the lowest possible value on exit.

┆Issue is synchronized with this Jira Story ┆friendlyId: TOIL-350

[stain]

Will it be possible to use other S3 backends than AWS?

[DailyDreaming]

@abatilo We have sprint planning tomorrow and I’m going to propose putting this into the upcoming sprint.