Multi stage model inference pipelines

/kind feature In many situations, inferenceservice is contained many small services. If we want to deploy this big servcie by KFServing, we should create a server to connect these small services together, and use custom model to deploy this big service. If we want to change the connection, we should recode the server and rebuild the image. If KFServing can handle this by edit the deploy yaml, it’ll save much developer time. Describe the solution you’d like [A clear and concise description of what you want to happen.]

                                                            +-----------------+
                                                            |   big-service   |
                                                            |                 |
                                                        +--->                 +-----+
          +---------------+        +----------------+   |   |    step 3-1/4   |     |      +-----------------+
          |  big-service  |        | big-service    |   |   |                 |     |      |    big-service  |
          |               |        |                |   |   +-----------------+     |      |                 |
          |               +------->+                +---+                           +----->+                 |
          |   step 1/4    |        |    step 2/4    |   |   +-----------------+     |      |                 |
          |               |        |                |   |   |   big-service   |     |      |    step 4/4     |
          +---------------+        +----------------+   |   |                 |     |      |                 |
                                                        +--->                 +-----+      +-----------------+
                                                            |   step 3-2/4    |
                                                            |                 |
                                                            +-----------------+

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]

[Nagarajj]

@yuzisun @cliveseldon what is the plan for Inference Graph, it is on roadmap but don’t see anything concrete. Would be good to get some clarity. Thanks.

[Iamlovingit]

@yuzisun Thanks for your reply. Some pattern recognition methods is splited some steps. For example, the face recognition:

1. find the face area
2. compute the features of these face
3. match the result from face database Usually, 1,2 are two different models. 1 output is 2 input.