Kubernetes Operator support

Description

Operators main usecase are to support users in automating install/setup/maintanence tasks outside the application. This proposal is NOT about having a generic quarkus operator or some auto-wiring operator required to setup services, but about making it easy for users to setup and write an operator for their own app.

There are two parts, and both will be optional:

1. have a way to generate a CRD derived from application properties + java code/annotations
2. provide a template or possibly even a extension to make it easy to write an operator in Quarkus

Analysis

proposal from @iocanel: Using operators is a very common way to provision middleware and service on Kubernetes. Be it Jenkins, Infinispan, MySQL, Kafka etc these are all things that nowadays get provisioned and operated using operators.

But I am interested mostly in operators developed by the user.

In many cases, it makes sense the users themselves to create operators to manage their application (especially the parts that involve state handling and can’t just be something as simple as starting a pod).

Examples I encountered in my previous Jobs:

Messaging gateway.

In one of my previous jobs I was implementing a custom SMS gateway for one of the global leading companies in mobile marketing. Back then for every single of the mobile company we connected (e.g. Vodafone, Tim, Wind), we had to go and manually perform an independent installation, where we configured things like:

• ip
• protocol
• connection pool sizes
• throttling
• contact rules
• user black listing / white listing
• routing information
• scaling information
• inbound traffic control (on/off switch)
• outbound traffic control (on/off switch)

Then we would have to keep track which server hosted what integration and manually perform tasks like, update, backup, restore, set offline, debug, network trace etc. Each time we needed to perform an upgrade to the backend we needed to perform sequences like:

1. Open the document which lists which server hosts integration with which mobile company.
2. Go to the target server console.
3. Close inbound traffic channels.
4. Wait until all pending messages are processed
5. Close outbound traffic channels.
6. Perform the backend upgrade.
7. Switch things back on.

Today we would have used Operators in order to automate this manual process.

Every single integration would be a Kubernetes resource like:

apiVersion: io.gateway/v1
kind: MobileOperatorConnection
metadata:
  name: tim-brazil
spec:
    ip: xxx.yyy.zzz
    poolSize:
        min: 5
        max: 10
    routing: 
        - shortcode: 1010
          app: super-promo-1
    contact-rules:
        maxPerDay: 5
   inboundChannel: on
   outboundChannel: on

From my CLI I would be able to see all the available integrations. I would be able to perform changes just by editing this resource and the operator would perform (1-7) for me. For example, if we needed to reroute shortcode 1010 to a different backend service say super-promo-2, we would just update a kubernetes resource and change super-promo-1 to super-promo-2 and the operator would do all those steps for me.

Insurance Company Underwriting.

Before my time in mobile marketing, I worked for a couple of years in the Insurance industry. One of our customers was an Insurance company. This company had different installations per region (each region was operated by different teams, different underwriters, was using different databases and had different rules). All the installations were connected to their central installation.

Software upgrades or changes in their underwriting policies meant that each installation needed to stop working until the central gets updated.

Do you see where this is going? Someone had to perform this by hand. Today, something like this would be trivial to implement using an operator.

Even though these operations are driven by business needs, they are implemented most of the time in golang because of its footprint. Quarkus gives us the opportunity to steer many of the people using golang to back java. So, I would like to make developing operators using Quarkus as easy as possible.

So what does one need to create an operator?

1. A custom resource definition (aka CRD)
2. A controller that monitors resources of that CRD and re-concilliates the state.

For step 1 we can provide a Java to CRD functionality like we did Java to WSDL years back. From there we can provide something like a framework for building operators on quarkus or even quarkify https://github.com/jvm-operators

Tasks

• add way to generate a “installation/setup/update” CRD for use in a operator for users app
• consider quarkifying https://github.com/jvm-operators
• add a framework/extension to make it easy to write a kubernetes operator

[gastaldi]

Agreed, this can now be closed in favor of the quarkus-operator-sdk extension. Thanks!

[metacosm]

Should we close this since we have the quarkus-operator-sdk extension now? I’d favor opening more targeted issues under that repository now.