Specifying an invalid zone causes infinite retries

I specified a zone of us-east1-a, which is not a legitimate zone. The problem is not handled well and the error message is confusing and unhelpful:

See operations/ELXWv-WdKxis4eaT-cfmx7EBINHtgZmgHSoPcHJvZHVjdGlvblF1ZXVl which begins with:

done: true
error:
  code: 5
  message: no zones available
.
.
.

Cromwell loops forever in this pattern (current develop head, 678712acb303a6cce0d35d6bcb963f19407439f5):

[INFO] [01/26/2017 16:30:48.889] [cromwell-system-akka.dispatchers.backend-dispatcher-31] [akka://cromwell-system/user/cromwell-service/$b/$a] The JES polling actor Actor[akka://cromwell-system/user/cromwell-service/$b/$a/$b#-1628247913] unexpectedly terminated while conducting 1 polls. Making a new one...
[INFO] [01/26/2017 16:30:48.889] [cromwell-system-akka.dispatchers.backend-dispatcher-31] [akka://cromwell-system/user/cromwell-service/$b/$a] watching Actor[akka://cromwell-system/user/cromwell-service/$b/$a/$c#1867219466]
[ERROR] [01/26/2017 16:30:58.241] [cromwell-system-akka.dispatchers.backend-dispatcher-31] [akka://cromwell-system/user/cromwell-service/$b/$a/$c] null
java.lang.NullPointerException
	at cromwell.backend.impl.jes.Run$.ceInfo$lzycompute$1(Run.scala:122)
	at cromwell.backend.impl.jes.Run$.ceInfo$1(Run.scala:122)
	at cromwell.backend.impl.jes.Run$.machineType$lzycompute$1(Run.scala:123)
	at cromwell.backend.impl.jes.Run$.machineType$1(Run.scala:123)
	at cromwell.backend.impl.jes.Run$.interpretOperationStatus(Run.scala:130)
	at cromwell.backend.impl.jes.statuspolling.JesPollingActor.interpretOperationStatus(JesPollingActor.scala:86)
	at cromwell.backend.impl.jes.statuspolling.JesPollingActor$$anon$1.onSuccess(JesPollingActor.scala:72)
	at cromwell.backend.impl.jes.statuspolling.JesPollingActor$$anon$1.onSuccess(JesPollingActor.scala:69)
	at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseAndCallback(BatchUnparsedResponse.java:197)
	at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseNextResponse(BatchUnparsedResponse.java:155)
	at com.google.api.client.googleapis.batch.BatchRequest.execute(BatchRequest.java:253)
	at cromwell.backend.impl.jes.statuspolling.JesPollingActor.runBatch(JesPollingActor.scala:67)
	at cromwell.backend.impl.jes.statuspolling.JesPollingActor.cromwell$backend$impl$jes$statuspolling$JesPollingActor$$handleBatch(JesPollingActor.scala:58)
	at cromwell.backend.impl.jes.statuspolling.JesPollingActor$$anonfun$receive$1.applyOrElse(JesPollingActor.scala:36)
	at akka.actor.Actor$class.aroundReceive(Actor.scala:496)
	at cromwell.backend.impl.jes.statuspolling.JesPollingActor.aroundReceive(JesPollingActor.scala:23)
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
	at akka.actor.ActorCell.invoke(ActorCell.scala:495)
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
	at akka.dispatch.Mailbox.run(Mailbox.scala:224)
	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[mcovarr]

us-east1-a is not a legitimate zone, but nevertheless Cromwell’s handling of this user error should be better.

https://cloud.google.com/compute/docs/regions-zones/regions-zones

[ruchim]

Running this on Cromwell 35 on PAPI v2 returns this error: Task w.t:NA:1 failed. The job was stopped before the command finished. PAPI error code 5. Execution failed: selecting zone: no regions/zones match request

AC: For both the PAPI v1/v2 backends, add more context to this error. Something along the lines of… Unable to start job because the zones defined in the runtime parameter zones: "$zones" doesn't match zones/regions supported by GCE. Please resubmit the job with a list of supported zones/regions by consulting a list of options here: https://cloud.google.com/compute/docs/regions-zones/