Docker mount problem on MAC

Output from a step is saved to a folder which by default is not allowed to be mounted by Docker. The folder is /var/folders. Default configuration includes: /Users, /Volumes, /private, /tmp. If add it manually in Docker preferences all works fine.

To run the test: cwl-runner --debug workflow.cwl job.yml Link to testing example: https://github.com/michael-kotliar/cwl_sandbox/tree/master/mac_docker_fail

Part from the log:

[job copy_2] /var/folders/sd/41rg42_16q72_2yzl_vvgsbw0000gn/T/tmpuEpjSV$ docker \
    run \
    -i \
    --volume=/var/folders/sd/41rg42_16q72_2yzl_vvgsbw0000gn/T/tmptNfjZl/./empty_copy.txt:/private/var/lib/cwl/stgcce3847f-a49b-4279-b1b1-0d5366ac8b4e/empty_copy.txt:ro \
    --volume=/private/var/folders/sd/41rg42_16q72_2yzl_vvgsbw0000gn/T/tmpuEpjSV:/private/var/spool/cwl:rw \
    --volume=/private/var/folders/sd/41rg42_16q72_2yzl_vvgsbw0000gn/T/tmpxbsJsT:/tmp:rw \
    --workdir=/private/var/spool/cwl \
    --read-only=true \
    --user=501 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/private/var/spool/cwl \
    ubuntu \
    cp \
    /private/var/lib/cwl/stgcce3847f-a49b-4279-b1b1-0d5366ac8b4e/empty_copy.txt \
    empty_copy_copy.txt
docker: Error response from daemon: Mounts denied: cker-for-mac/osxfs/#namespaces for more info.
.
l_vvgsbw0000gn/T/tmptNfjZl/./empty_copy.txt
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/do.
ERRO[0000] error getting events from daemon: net/http: request canceled 

Docker version:

Client:
 Version:      17.03.0-ce
 API version:  1.26
 Go version:   go1.7.5
 Git commit:   60ccb22
 Built:        Thu Feb 23 10:40:59 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.03.0-ce
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   3a232c8
 Built:        Tue Feb 28 07:52:04 2017
 OS/Arch:      linux/amd64
 Experimental: true

cwl-runner version: 1.0.20170224141733

[denis-yuen]

This sounds similar to something we encountered. There’s a possible workaround at the bottom of https://dockstore.org/docs/faq

Docker behaves a bit differently on a mac than on a typical ubuntu machine. By default the only shared > volumes are /Users, /Volumes, /tmp, and /private. Note that /var is not a shared directory (and can’t be > set as one). CWLtool uses your TMPDIR (the env variable) to setup volumes with docker, which on a Mac can default to a subdirectory of /var. In order to get CWLtool working on your mac, you need to set your TMPDIR to be under one of the shared volumes in Docker for Mac. You can do this by doing something similar to the following:

export TMPDIR=/tmp/docker_tmp

[wltrimbl]

I’m getting this error for certain data sources on OSX.

Expected Behavior

Toy workflow to download a file using http and display it using a docker image.

Actual Behavior

Fails with osx-specific error, related to Issue #328 and #812;

The path /var/folders/2g/k5ylcjt57_d4bqw352_flw740000gn/T/tmpaeCr3G is not shared from OS X and is not known to Docker.

input file is downloaded and staged in

/var/folders/2g/k5ylcjt57_d4bqw352_flw740000gn/T/tmpaeCr3G

but docker needs this to be prepended with /private on some OSX systems.

The remedy in Issue #328

export TMPDIR="/private${TMPDIR}" 

makes the workflow complete, and it works on ubuntu.

Workflow Code

cat > cat.cwl <<"EOF"
#!/usr/bin/env cwl-runner --debug
cwlVersion: cwl:v1.0
class: CommandLineTool
baseCommand: cat
hints:
  - class: DockerRequirement
    dockerPull: ubuntu:16.04
inputs:
  - id: input_file
    type: File
    inputBinding:
      position: 1
      valueFrom: $(inputs.input_file)
outputs: []
EOF

cat > cat-job.yml <<"EOF"
cwlVersion: cwl:v1.0
input_file:
  class: File
  location: "https://raw.githubusercontent.com/common-workflow-language/cwltool/master/README.rst"
  basename: README.Rst
EOF

Output

cwl-runner cat.cwl cat-job.yml

/Users/username/anaconda3/envs/python2/bin/cwl-runner 1.0.20180822145714
Resolved 'cat.cwl' to 'file:///Users/username/ks/cat.cwl'
[job cat.cwl] /private/tmp/docker_tmpZhORn3$ docker \
    run \
    -i \
    --volume=/private/tmp/docker_tmpZhORn3:/var/spool/cwl:rw \
    --volume=/private/var/folders/2g/k5ylcjt57_d4bqw352_flw740000gn/T/tmp2wE_eI:/tmp:rw \
    --volume=/var/folders/2g/k5ylcjt57_d4bqw352_flw740000gn/T/tmpaeCr3G:/var/lib/cwl/stg5c087dc6-b021-4e62-8adb-9198787d0626/README.Rst:ro \
    --workdir=/var/spool/cwl \
    --read-only=true \
    --user=501:20 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/var/spool/cwl \
    ubuntu:16.04 \
    cat
docker: Error response from daemon: Mounts denied: 
The path /var/folders/2g/k5ylcjt57_d4bqw352_flw740000gn/T/tmpaeCr3G
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.
.
time="2018-08-24T09:38:20-05:00" level=error msg="error waiting for container: context canceled"
[job cat.cwl] completed permanentFail
Final process status is permanentFail
```

## Your Environment
* cwltool version: 
cwltool 1.0.20180822145714
OSX 10.13.6