Docker buildimage intermittently builds image

Hello, I am playing around with prefect and trying to use the buildimage for docker

from prefect import task, Flow
from prefect.tasks.docker import images


@task
def build_image():
    images.BuildImage(path='.').run()


with Flow("ETL") as flow:
    e = build_image()

flow.run()

[2019-07-16 19:36:37,341] INFO - prefect.FlowRunner | Beginning Flow run for 'ETL'
[2019-07-16 19:36:37,341] INFO - prefect.FlowRunner | Starting flow run.
[2019-07-16 19:36:37,342] INFO - prefect.TaskRunner | Task 'build_image': Starting task run...
[2019-07-16 19:36:41,274] INFO - prefect.TaskRunner | Task 'build_image': finished task run for task with final state: 'Success'
[2019-07-16 19:36:41,275] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded

I noticed that this intermittently builds an image once about ever 4-5 times I run it. Anyone have an idea what is going on?

[richard-avelar]

Sounds good to me! Thanks for prompt feedback!

[cicdw]

Great!

So I have a guess here -> it’s possible that client.build is an asynchronous / deferred call of some kind, and whenever the Prefect Task exits it prevents the build from actually occurring. What we should do is update the Prefect Task to actually iterate over the client.build generator to ensure it completes before returning from the Task.

In the meantime, I recommend you implement a custom Task like so:

import docker


@task
def build_image(path):
    client = docker.APIClient(base_url="unix:///var/run/docker.sock", version="auto")
    client.build(path=path, tag=None, nocache=False, rm=True, forcerm=False)
    response = [line for line in client.build(path=".", tag=None, nocache=False, rm=True, forcerm=False)]
    return 

And I’d bet this works. Looks like I need to audit our Docker Tasks for this behavior!