[bug] Pipeline metrics Invalid input error: Unknown execution spec

Environment

• How do you deploy Kubeflow Pipelines (KFP)? Kubeflow/manifest v1.6 AKS v1.22

• KFP version: ml-pipeline/api-server:2.0.0-alpha.3

• KFP SDK version: ‘1.8.13’

Steps to reproduce

from kfp.components import InputPath, OutputPath, create_component_from_func
from kfp import dsl
def produce_metrics(
  # Note when the `create_component_from_func` method converts the function to a component, the function parameter "mlpipeline_metrics_path" becomes an output with name "mlpipeline_metrics" which is the correct name for metrics output.
  mlpipeline_metrics_path: OutputPath('Metrics'),
):
    import json

    accuracy = 0.1
    metrics = {
    'metrics': [{
      'name': 'accuracy-score', # The name of the metric. Visualized as the column name in the runs table.
      'numberValue':  accuracy, # The value of the metric. Must be a numeric value.
      'format': "PERCENTAGE",   # The optional format of the metric. Supported values are "RAW" (displayed in raw format) and "PERCENTAGE" (displayed in percentage format).
    }]
    }
    with open(mlpipeline_metrics_path, 'w') as f:
        json.dump(metrics, f)

produce_metrics_op = create_component_from_func(
    produce_metrics,
    base_image='python:3.7',
    packages_to_install=[],
    output_component_file='component.yaml',
)
from kfp import dsl
import os, kfp
@dsl.pipeline(
    name='KF iris PoC',
    description='Simple TF'
)
def test():
     task=produce_metrics_op()
import os,kfp
with open(os.environ['KF_PIPELINES_SA_TOKEN_PATH'], "r") as f:
    TOKEN = f.read()
client = kfp.Client(host='http://ml-pipeline.kubeflow.svc.cluster.local:8888',existing_token=TOKEN) 

client.create_run_from_pipeline_func(
    test,
    mode=kfp.dsl.PipelineExecutionMode.V1_LEGACY,
    arguments={}
   
)

Expected result

Materials and reference

https://www.kubeflow.org/docs/components/pipelines/sdk/pipelines-metrics/

I cannot see the pipeline metrics in the UI but the pipeline artifact is logged i am getting this error in ml-pipeline

I0912 09:12:02.846997       8 error.go:259] Invalid input error: Unknown execution spec
InternalServerError: failed to unmarshal workflow

Labels

---

Impacted by this bug? Give it a 👍.

[chensun]

can we pin back to KFP 2.0.0a2 in KF 1.6? I assume that version is possibly tested more thoroughly with KF 1.6 RC releases.

I take this back, checked the change history and it appears we never included KFP 2.0.0a2 in any KF 1.6 RC release. Let me focus on this specific issue first.

Discussed with @gkcalat offline. We’ll release with 2.0.0a5 with the fix, and tentatively use this version for KF 1.6 patch release. That being said, I still think in future KF release, we shouldn’t automatically pick the latest KFP alpha release available.

[chensun]

Thanks @chensun, do you know when we can expect 2.0.0.a5?

@annajung 2.0.0a5 release is done: https://github.com/kubeflow/pipelines/releases/tag/2.0.0-alpha.5