InferenceService Status restructuring

Currently, ModelMesh has its own set of status fields that it populates in the InferenceService status since we allow unknown fields there. Similarly, KServe has its own status fields that are predominantly based on knative fields.

The goal here is to converge on a common set of status fields that would work with the various different deployment modes.

As an overview, here are the current status fields:

InferenceServiceStatus

InferenceServiceStatus
  ├──── duckv1.Status `json:",inline"`
  │        │    // The 'Generation' of the Service that was last processed by the controller.
  │        ├─── ObservedGeneration int64 `json:"observedGeneration,omitempty"`
  │        │
  │        │    // Conditions the latest available observations of a resource's current state.
  │        |    // Condition defines a readiness condition for a Knative resource.
  │        ├─── Conditions Conditions `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"`
  │        | 
  │        |     // Additional Status fields for the Resource to save some additional State as well as convey more information to the user.
  │        ├─── Annotations map[string]string `json:"annotations,omitempty"`
  │
  ├──── Address *duckv1.Addressable `json:"address,omitempty"`
  ├──── URL *apis.URL `json:"url,omitempty"`
  ├──── Components map[ComponentType]ComponentStatusSpec `json:"components,omitempty"`

ComponentTypes are predictor, explainer, transformer. Which map to a ComponentStatusSpec.

An example of an actual status can be found here: https://pastebin.com/wkCrZyxk

ModelMesh Predictor Status

PredictorStatus
  ├──── Available bool `json:"available"`
  │
  │     // One of 'UpToDate', 'InProgress', 'BlockedByFailedLoad', or 'InvalidSpec'
  ├──── TransitionStatus TransitionStatus `json:"transitionStatus"
  │
  │     // High level state string: Pending, Standby, Loading, Loaded, FailedToLoad
  ├──── ActiveModelState ModelState `json:"activeModelState"
  ├──── TargetModelState ModelState `json:"targetModelState"`
  │
  │     // Details of last failure, when load of target model is failed or blocked
  ├──── LastFailureInfo *FailureInfo `json:"lastFailureInfo,omitempty"`
  │
  │     // Addressable endpoint for the deployed trained model. This will be "static" and will not change when the model is mutated
  ├──── HTTPEndpoint string `json:"httpEndpoint"`
  ├──── GrpcEndpoint string `json:"grpcEndpoint"`
  │
  │     // How many copies of this predictor's models failed to load recently
  ├──── FailedCopies int `json:"failedCopies"`

An example of an actual predictor status can be found here: https://pastebin.com/wBM3WgFW

---

Action item: Decide how we can restructure the fields in a way that make sense for all the deployment modes. Do we still rely on the Knative types?

[chinhuang007]

For both KServe and ModelMesh to update and understand the status/state of an inference service, I would suggest to continue to use the Knative types. That probably means that InferenceServiceStatus needs to cover ModelMesh status requirements, and ModelMesh needs to include Knative packages and change code accordingly.

[pvaneck]

FYI @yuzisun , @njhill , @chinhuang007