Can't send environment to pebble

I’m working on converting a charm to use pebble (as a proof of concept) and am running into problems trying to send it an environment configuration.

The pebble docs specify:

        environment:
            - VAR1: val1
            - VAR2: val2
            - VAR3: val3

So I tried passing a list of dictionaries to the environment config, but I got:

unit-gunicorn-0: 10:04:56 INFO unit.gunicorn/0.juju-log About to dump yaml config <<EOM
description: gunicorn layer
services:
  gunicorn:
    command: /srv/gunicorn/run
    default: start
    environment:
    - FAVOURITEFOOD: burgers
    - FAVOURITEDRINK: ale
    override: replace
    summary: gunicorn service
summary: gunicorn layer

EOM
unit-gunicorn-0: 10:04:56 ERROR unit.gunicorn/0.juju-log Uncaught exception while in charm code:
Traceback (most recent call last):
  File "./src/charm.py", line 438, in <module>
    main(GunicornK8sCharm, use_juju_for_storage=True)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/main.py", line 406, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/main.py", line 140, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/framework.py", line 278, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/framework.py", line 722, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/framework.py", line 767, in _reemit
    custom_handler(event)
  File "./src/charm.py", line 137, in _on_gunicorn_workload_ready
    container.add_layer("gunicorn", pebble_config)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/model.py", line 1065, in add_layer
    self._pebble.add_layer(label, layer, combine=combine)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/pebble.py", line 668, in add_layer
    layer_yaml = Layer(layer).to_yaml()
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/pebble.py", line 427, in __init__
    self.services = {name: Service(name, service)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/pebble.py", line 427, in <dictcomp>
    self.services = {name: Service(name, service)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/pebble.py", line 463, in __init__
    self.environment = dict(raw.get('environment') or {})
ValueError: dictionary update sequence element #0 has length 1; 2 is required

Looking at the code for the operator framework self.environment = dict(raw.get('environment') or {}) so I tried just passing in a dictionary, but I got the following:

unit-gunicorn-0: 10:18:33 ERROR juju.worker.uniter pebble poll failed: hook failed
unit-gunicorn-0: 10:18:37 INFO unit.gunicorn/0.juju-log About to dump yaml config <<EOM
description: gunicorn layer
services:
  gunicorn:
    command: /srv/gunicorn/run
    default: start
    environment:
      FAVOURITEDRINK: ale
      FAVOURITEFOOD: burgers
    override: replace
    summary: gunicorn service
summary: gunicorn layer

EOM
unit-gunicorn-0: 10:18:37 ERROR unit.gunicorn/0.juju-log Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/pebble.py", line 531, in _request
    response = self.opener.open(request, timeout=self.timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./src/charm.py", line 450, in <module>
    main(GunicornK8sCharm, use_juju_for_storage=True)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/main.py", line 406, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/main.py", line 140, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/framework.py", line 278, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/framework.py", line 722, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/framework.py", line 767, in _reemit
    custom_handler(event)
  File "./src/charm.py", line 150, in _on_gunicorn_workload_ready
    container.add_layer("gunicorn", pebble_config)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/model.py", line 1065, in add_layer
    self._pebble.add_layer(label, layer, combine=combine)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/pebble.py", line 682, in add_layer
    self._request('POST', '/v1/layers', body=body)
  File "/var/lib/juju/agents/unit-gunicorn-0/charm/venv/ops/pebble.py", line 542, in _request
    raise APIError(body, code, status, message)
ops.pebble.APIError: cannot parse layer YAML: cannot parse layer "gunicorn": yaml: unmarshal errors:
  line 7: cannot unmarshal !!map into []plan.StringVariable
unit-gunicorn-0: 10:18:38 ERROR juju.worker.uniter.operation hook "gunicorn-workload-ready" (via hook dispatching script: dispatch) failed: exit status 1

[benhoyt]

This is fixed now in the latest Pebble and Python Operator Framework. Pebble accepts and returns an array of objects, like so:

    environment:
      - FAVOURITEFOOD: burgers
      - FAVOURITEDRINK: ale

And in Python-land in the Service.environment attribute, it’s serialized to a list of 2-tuples, like so:

[('FAVOURITEFOOD', 'burgers'), ('FAVOURITEDRINK', 'ale')]

Like your food preferences, by the way. 😉

[niemeyer]

The issue with plain objects is that environment variables may have order depending on how we allow them to expand. That’s why it’s a sequence. We can switch to a map and parse that in Go taking the order into account, but that’s not a feature that follows the spec. We’ve done that before, so we can also choose to do it here again if there’s consensus, but we need to at least verify that Python would be able to parse it correctly, for example.

Otherwise, you’re on the right track with the marshaler.