Cannot pickle objects when using unix socket

One of my use-cases is loading a list of containers in one process and then pushing them to be processed in another process, using a standard python queue.

While this method works well with the tcp:// socket, when using the unix:// socket the pickler will throw the following exception:

    queue.put((QUEUE_OBJ, obj), block=block)
  File "<string>", line 2, in put
  File "/usr/local/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
    conn.send((self._id, methodname, args, kwds))
  File "/usr/local/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/local/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'UnixAdapter.__init__.<locals>.<lambda>'

On one hand I could just push the IDs of the objects downstream, but I need some of the attributes belonging to the Containers as well, and it makes for quite an inconvenient way of sharing them.

I am not sure if that is the expected behaviour; but the TCP protocol works well enough and I can’t see a reason why UNIX should not.

[drzraf]

maintainers? I’m still annoyed of not being able to work on docker-py containers within concurrent.

[jason-kane]

Not being able to use multiprocess w/docker api calls is lame. Not doing it out of some misguided stubborn refusal when the fix is near-trivial is ridiculous.