How to specify the env values in ComponentSpec to build reusable components

What steps did you take:

I am building a reusable component say alert-manager which will be used across every pipeline. The docker images used to build the component needs some environments variables to be set. As per the documentation (https://www.kubeflow.org/docs/pipelines/reference/component-spec/) , env can be set as maps. So I have added env params as shown below

name: dsw-alert
description: sends events to prometheus alert manager

inputs:
- {name: alert_name, type: String, description: 'alert name'}
- {name: severity, type: String, description: 'severity of the notification'}

implementation:
  container:
    image: dsw-alert-manager:1.0.0-10
    args: [
    "src/main.py",
    "--alert_name", {inputValue: alert_name},
    "--severity", {inputValue: severity}]
    env:
      - name: ALERT_MANAGER_HOST
        value: 'http://alert-manager.fms-svc.svc.cluster.local'

What happened:

When I am trying to load the component using load_component_from_file(“data-loader-qa.yaml”) dsl-compile of the piepline fails with below error (without the env section in reusable alert manager component, I am able to compile the pipeline component).

Error: ContainerSpec.from_dict(struct=OrderedDict([('image', 'dsw-alert-manager:1.0.0-10'), ('args', ['src/main.py', '--alert_name', OrderedDict([
('inputValue', 'alert_name')]), '--severity', OrderedDict([('inputValue', 'severity')])]), ('env', [OrderedDict([('name', 'ALERT_MANAGER_HOST'), ('value', 'http://alert-manager.fms-svc.sv
c.cluster.local')])])])) failed with exception:
Error: Structure "[OrderedDict([('name', 'ALERT_MANAGER_HOST'), ('value', 'http://alert-manager.fms-svc.svc.cluster.local')])]" is incompatible with type "typing.Mapping[str, str]" - it d
oes not have dict type.
Error: Structure "[OrderedDict([('name', 'ALERT_MANAGER_HOST'), ('value', 'http://alert-manager.fms-svc.svc.cluster.local')])]" is not None.

Error: Structure "[OrderedDict([('name', 'ALERT_MANAGER_HOST'), ('value', 'http://alert-manager.fms-svc.svc.cluster.local')])]" is incompatible with type "typing.Union[typing.Mapping[str,
 str], NoneType]" - none of the types in Union are compatible.
Error: GraphImplementation.from_dict(struct=OrderedDict([('container', OrderedDict([('image', 'harbor-registry-mndc.uidai.gov.in/fms-qa/dsw-alert-manager:1.0.0-10'), ('args', ['src/main.p
y', '--alert_name', OrderedDict([('inputValue', 'alert_name')]), '--severity', OrderedDict([('inputValue', 'severity')])]), ('env', [OrderedDict([('name', 'ALERT_MANAGER_HOST'), ('value',
 'http://alert-manager.fms-svc.svc.cluster.local')])])]))])) failed with exception:
__init__() got an unexpected keyword argument 'container'


Error: Structure "OrderedDict([('container', OrderedDict([('image', 'dsw-alert-manager:1.0.0-10'), ('args', ['src/main.py', '--alert_name', Ordere
dDict([('inputValue', 'alert_name')]), '--severity', OrderedDict([('inputValue', 'severity')])]), ('env', [OrderedDict([('name', 'ALERT_MANAGER_HOST'), ('value', 'http://alert-manager.fms
-svc.svc.cluster.local')])])]))])" is incompatible with type "typing.Union[kfp.components._structures.ContainerImplementation, kfp.components._structures.GraphImplementation, NoneType]" -
 none of the types in Union are compatible.

What did you expect to happen:

How to specify the environmental variables in reusable component spec? Any input on this is appreciated.

Environment:

How did you deploy Kubeflow Pipelines (KFP)?

Custom manifests deployment on on-prem kunernetes cluster with multi user support.

KFP version: 1.2.0

KFP SDK version: 1.1.1

/kind bug

[Bobgy]

/cc @chensun @Ark-kun

Hi @ShilpaGopal, mounting secrets are probably never reusable, because they depend on your cluster setup. Therefore, you can only override env vars after initiating a component as an instance. See https://github.com/kubeflow/pipelines/blob/cadcac08bd6e2712ce62d7eb59ff0b3f2ee1bbe2/samples/core/visualization/tensorboard_minio.py#L168-L181

[yueno06]

@Bobgy Excuse me for asking a question in a closed issue.

TODO: update the mentioned doc that env is not supported

To follow the comments above, is env no longer supported?

I’m confused because there is a statement that env can be configured in implementation