[bug] Cannot find how to type pipeline parameters for JsonArray or JsonObject

What steps did you take

I’m trying to utilize the ModelBatchPredictOp from the GCP Components. There are two fields, one with a type of JsonObject and the other with the type JsonArray. The pipeline accepts the fields for this operator as input parameters to the pipeline and passes them to the operator in the constructor. Since they’re in the pipeline params, they must be typed. However, I am unable to find any type configuration that coerces to JsonArray or JsonObject, and I cannot find any documentation on how to do this.

How do I pass these values?

For example, gcs_source_uris is supposed to be a JsonArray. When passing this through as a pipeline param typed to List[str] it fails to compile with this error:

kfp.dsl.types.InconsistentTypeException: Incompatible argument passed to the input "gcs_source_uris" of component "model_batch_predict": Argument type "typing.List[str]" is incompatible with the input type "JsonArray"

@kfp.v2.dsl.pipeline(
    name="batch-predict-v0",
    pipeline_root=pipeline_gcs_root_path)
def batch_predict_pipeline(
                           ...
                           gcs_source_uris: List[str],
                           ...
                           ):


        predict_op = ModelBatchPredictOp(
            ...
            gcs_source_uris=gcs_source_uris,
            ....
        )

I understand the error, and I’ve exhausted any other potential types I could think of. I’ve also tried to find any references to JsonObject or JsonArray in the SDK and none of them reference any Python classes I can import to type these parameters as. I’m gathering this is some kind of protobuf type, but when trying to use that, it still fails. Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[Ark-kun]

@dboshardy

Python’s list type is mapped to JsonArray Python’s dict type is mapped to JsonObject.

kfp.dsl.types.InconsistentTypeException: Incompatible argument passed to the input “gcs_source_uris” of component “model_batch_predict”: Argument type “typing.List[str]” is incompatible with the input type “JsonArray”

typing.List[str] should work the same as list, but there might be newly introduced bugs in v2 that break that.

For anyone who comes across this, jsonobject is the package that contains the classes required. This should be in the documentation somewhere.

The jsonobject package has no relation to the JsonObject type name in KFP components.

[Luci1306]

I have a question: how can we use the for loop for gcs_source_uris at batch_predict_pipeline, not inside the component which was passed. Thanks