Having issues running on Google Cloud Genomics

I’m attempting to launch DeepVariant on GCP, with the VerilyGRCH38 ref genome and the whole exome model. Unfortunately the job never finishes. Any pointers on how to debug this would be much appreciated.

#!/bin/bash
set -euo pipefail
# Set common settings.
PROJECT_ID=valis-194104
OUTPUT_BUCKET=gs://canis/CNR-data
STAGING_FOLDER_NAME=deep_variant_files
OUTPUT_FILE_NAME=TLE_a_001_deep_variant.vcf
# Model for calling whole exome sequencing data.
MODEL=gs://deepvariant/models/DeepVariant/0.7.0/DeepVariant-inception_v3-0.7.0+data-wes_standard
IMAGE_VERSION=0.7.0
DOCKER_IMAGE=gcr.io/deepvariant-docker/deepvariant:"${IMAGE_VERSION}"
COMMAND="/opt/deepvariant_runner/bin/gcp_deepvariant_runner \
  --project ${PROJECT_ID} \
  --zones us-west1-b \
  --docker_image ${DOCKER_IMAGE} \
  --outfile ${OUTPUT_BUCKET}/${OUTPUT_FILE_NAME} \
  --staging ${OUTPUT_BUCKET}/${STAGING_FOLDER_NAME} \
  --model ${MODEL} \
  --regions gs://canis/CNR-data/CDS-canonical.bed \
  --bam gs://canis/CNR-data/TLE_a_001.bam \
  --bai gs://canis/CNR-data/TLE_a_001.bam.bai \
  --ref gs://genomics-public-data/references/GRCh38_Verily/GRCh38_Verily_v1.genome.fa  \
  --gcsfuse"
# Run the pipeline.
gcloud alpha genomics pipelines run \
    --project "${PROJECT_ID}" \
    --service-account-scopes="https://www.googleapis.com/auth/cloud-platform" \
    --logging "${OUTPUT_BUCKET}/${STAGING_FOLDER_NAME}/runner_logs_$(date +%Y%m%d_%H%M%S).log" \
    --zones us-west1-b \
    --docker-image gcr.io/deepvariant-docker/deepvariant_runner:"${IMAGE_VERSION}" \
    --command-line "${COMMAND}"

I get the following error:

      Traceback (most recent call last):
        File "/opt/deepvariant_runner/src/gcp_deepvariant_runner.py", line 862, in <module>
          run()
        File "/opt/deepvariant_runner/src/gcp_deepvariant_runner.py", line 845, in run
          _run_make_examples(pipeline_args)
        File "/opt/deepvariant_runner/src/gcp_deepvariant_runner.py", line 340, in _run_make_examples
          _wait_for_results(threads, results)
        File "/opt/deepvariant_runner/src/gcp_deepvariant_runner.py", line 352, in _wait_for_results
          result.get()
        File "/usr/lib/python2.7/multiprocessing/pool.py", line 572, in get
          raise self._value
      RuntimeError: Job failed with error "run": operation "projects/valis-194104/operations/13939489157244551677" failed: executing pipeline: Execution failed: action 5: unexpected exit status 1 was not ignored (reason: FAILED_PRECONDITION)
    details:

[pichuan]

@nmousavi from the Google Cloud team to take a look. I’ll also take a look later today when I find some more time.

[nmousavi]

Lets first see if that’s the case (i.e. having bed file in public bucket resolves it). If yes, this is a bug and we will fix it.