[feature] Allow users to skip `docker build` in `kfp components build`

Feature Area

/area sdk

What feature would you like to see?

Building containerised components using the Python DSL, and packaging them using kfp components build is a really nice user experience - you get the benefits of the Python DSL without some of the drawbacks of “lightweight” components (e.g. having to install dependencies at runtime).

However, there are some scenarios where building container images using Docker is not an option, for example:

• Some organisations do not allow Docker to be installed on Data Scientists’ laptops (very common for enterprise users)
• Some CI/CD platforms do not support docker-in-docker, and use other mechanisms (such as kaniko) for building container images

This feature would allow the user to skip the actual container image build, but still perform the other functions in kfp components build (generation of the different files). The user would then be responsible for building the container images using their preferred mechanism (e.g. Cloud Build, Kaniko)

What is the use case or pain point?

Currently not all users are able to use the kfp components build command as they cannot run Docker (locally or in CI/CD)

Is there a workaround currently?

Current workarounds:

• Roll your own containerised components without the convenience of the kfp components build command
• I suppose you could mock the docker library as is done in the tests? But this is quite hacky!

---

Love this idea? Give it a 👍.

[browningjp-datatonic]

@connor-mccarthy no problem. The customer is not able to install Python packages from GitHub directly (everything goes through a private PyPi mirror), but we can make a start on development in a less restrictive environment.

Thanks for all your help!

[connor-mccarthy]

I think kfp components build --no-build-image . && gcloud builds submit ... is a good approach.

We probably don’t want to be in the business of maintaining integrations with many different build providers and the KFP SDK provides only a very thin wrapper around the commands you shared.

For example: kfp components build --no-build-image . && gcloud builds submit --cloudbuild-project ... becomes kfp components build --no-build-image --cloudbuild-project ... .. The diff is gcloud builds submit. There is not much typing saved and it comes at the cost of (1) transparency about what is going on and (2) debuggability for the user via an added layer of indirection.

Thank you for bringing this up @browningjp-datatonic! I will review #8387.