Issue with Docker storage - Cannot instantiate 'WindowsPath' on your system

Opened from the Prefect Public Slack Community

arthur: Hi, my first message here 🙂. I am new to Prefect, and I was able to run my flow from my local PC (Windows, python 3.8). Now I am trying to build it to a Docker Storage, and I am getting this error during the health check:

raise NotImplementedError("cannot instantiate %r on your system" NotImplementedError: cannot instantiate 'WindowsPath' on your system

Any ideas?

arthur: full error message is here:

Step 14/14 : RUN python /opt/prefect/healthcheck.py '["/opt/prefect/flows/project01-reports.prefect"]' '(3, 8)'
 ---> Running in 52f09f649e73
Beginning health checks...
System Version check: OK
Traceback (most recent call last):
  File "/opt/prefect/healthcheck.py", line 147, in <module>
    flows = cloudpickle_deserialization_check(flow_file_paths)
  File "/opt/prefect/healthcheck.py", line 40, in cloudpickle_deserialization_check
    flows.append(cloudpickle.load(f))
  File "/usr/local/lib/python3.8/pathlib.py", line 1043, in __new__
    raise NotImplementedError("cannot instantiate %r on your system"
NotImplementedError: cannot instantiate 'WindowsPath' on your system

Removing intermediate container 52f09f649e73
The command '/bin/sh -c python /opt/prefect/healthcheck.py '["/opt/prefect/flows/project01-reports.prefect"]' '(3, 8)'' returned a non-zero code: 1
Traceback (most recent call last):
  File "flows/project01/project01_flow.py", line 400, in <module>
    my_flow.register("project01")
  File "D:\Programas\py_temp\prefect\lib\site-packages\prefect\core\flow.py", line 1620, in register
    registered_flow = client.register(
  File "D:\Programas\py_temp\prefect\lib\site-packages\prefect\client\client.py", line 734, in register
    serialized_flow = flow.serialize(build=build)  # type: Any
  File "D:\Programas\py_temp\prefect\lib\site-packages\prefect\core\flow.py", line 1458, in serialize
    storage = self.storage.build()  # type: Optional[Storage]
  File "D:\Programas\py_temp\prefect\lib\site-packages\prefect\environments\storage\docker.py", line 351, in build
    self._build_image(push=push)
  File "D:\Programas\py_temp\prefect\lib\site-packages\prefect\environments\storage\docker.py", line 417, in _build_image
    raise ValueError(
ValueError: Your docker image failed to build!  Your flow might have failed one of its deployment health checks - please ensure that all necessary files and dependencies have been included.

znicholasbrown: Hi <@U01DENWR97F>, welcome! 👋

znicholasbrown: This sounds like it’s related to the way Core is pickling your flow as it’s trying to build - I’m going to open an issue on the Core repo because they’ll be better able to triage this.

znicholasbrown: <@ULVA73B9P> open “Issue with Docker storage - Cannot instantiate ‘WindowsPath’ on your system”

Original thread can be found here.

[arthur-datalab]

Hi, thanks for your feedback. I gave up on Windows and decided to install WSL2 with a Ubuntu distro on my Windows 10, then I could finally register Prefect without problems using the Pathlib python lib.

[cicdw]

@arthur-datalab I believe this problem is arising because pathlib.Path objects are not serializable (pickleable) between systems (in your case, Windows <-> Linux). A few things you might try:

• set stored_as_script=True on your Docker storage object; this will prevent the serialization of the Path object
• refactor your code to not use pathlib.Path