question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Windows pathing error when using Dockerfile method for Docker storage

See original GitHub issue

Description

When using the Dockerfile method to create storage and then register a flow with prefect cloud, there is an error COPY step that copies the flow into the Docker image.

C:\dev\scratch>python register.py
[2020-07-31 20:35:20] INFO - prefect.Docker | Building the flow's Docker storage...
Step 1/10 : FROM python:3.8.3
 ---> 659f826fabf4
Step 2/10 : RUN pip install prefect
 ---> Using cache
 ---> 79ef06e51b9f
Step 3/10 : RUN pip install pip --upgrade
 ---> Using cache
 ---> a0b4e4dec623
Step 4/10 : RUN pip show prefect || pip install git+https://github.com/PrefectHQ/prefect.git@0.12.6#egg=prefect[kubernetes]
 ---> Using cache
 ---> 00faab6426bd
Step 5/10 : RUN pip install boto3 wheel
 ---> Using cache
 ---> 7dd663167bd5
Step 6/10 : RUN mkdir -p /opt/prefect/
 ---> Using cache
 ---> ffca70689d93
Step 7/10 : COPY C:\dev\scratch\tmpk7_nwvnd\prefect-windows-pathing-repro.flow /opt/prefect/flows/prefect-windows-pathing-repro.prefect
COPY failed: stat /var/lib/docker/tmp/docker-builder087651641/C:devscratchtmpk7_nwvndprefect-windows-pathing-repro.flow: no such file or directory
Traceback (most recent call last):
  File "register.py", line 16, in <module>
    flow.register(
  File "C:\dev\scratch\venv\lib\site-packages\prefect\core\flow.py", line 1575, in register
    registered_flow = client.register(
  File "C:\dev\scratch\venv\lib\site-packages\prefect\client\client.py", line 677, in register
    serialized_flow = flow.serialize(build=build)  # type: Any
  File "C:\dev\scratch\venv\lib\site-packages\prefect\core\flow.py", line 1429, in serialize
    storage = self.storage.build()  # type: Optional[Storage]
  File "C:\dev\scratch\venv\lib\site-packages\prefect\environments\storage\docker.py", line 327, in build
    self._build_image(push=push)
  File "C:\dev\scratch\venv\lib\site-packages\prefect\environments\storage\docker.py", line 391, in _build_image
    raise ValueError(
ValueError: Your docker image failed to build!  Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.

This appears to be due to the fact that a Windows path is assumed to be a Posix path, so the backslashes. See above in “Step 7/10.”

Expected Behavior

The registration process should build the image without error, and finish the docker push and prefect registration processes.

Reproduction

https://github.com/aroder/prefect-windows-pathing-repro

Environment

Reproduced on 2 different Windows machines. Here are specs for one:

image

image

Optionally run prefect diagnostics from the command line and paste the information here. -->

>prefect diagnostics
{
  "config_overrides": {
    "cloud": {
      "use_local_secrets": true
    },
    "context": {
      "secrets": false
    }
  },
  "env_vars": [],
  "system_information": {
    "platform": "Windows-10-10.0.19041-SP0",
    "prefect_version": "0.12.6",
    "python_version": "3.8.3"
  }
}

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8

github_iconTop GitHub Comments

1reaction
sbreidbach-edrcommented, Aug 4, 2020

Worked like magic @joshmeek .

0reactions
joshmeekcommented, Aug 4, 2020

Yeah @sbreidbach-edr that appears to be related. Do you think you could also install off of master and give it another shot like @aroder did?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Error when mounting host path with Docker for Windows ...
The solution seems to be to add the Users group to the .docker folder and grant full access. Just the new local account...
Read more >
Docker Windows build fails with error: "The system cannot find ...
Cygwing uses hardlinks a lot. It all went wwll until Docker tried to commit an image and it failed with "hcsshim::ImportLayer - cannot...
Read more >
Troubleshoot topics | Docker Documentation
If you are using mounted volumes and get runtime errors indicating an application file is not found, access to a volume mount is...
Read more >
Troubleshoot volume errors - Docker Documentation
This topic discusses errors which may occur when you use Docker volumes or bind mounts. Error: Unable to remove filesystem. Some container-based utilities, ......
Read more >
About storage drivers - Docker Documentation
Each storage driver handles the implementation differently, but all drivers use stackable image layers and the copy-on-write (CoW) strategy. Note. Use Docker ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found