Framework does not log re-emission of events

If an event occurs, triggers a hook, but that this hook is deferred, only the first occurrence of the hook is logged. From the logs, we can’t even tell if the hook is being re-emitted at all.

In this example, an API call failure (on purpose) causes the on_start hook to be deferred.

    def on_start(self, event):
        """Occurs upon start or installation of the charm."""
        logging.info('Setting the pod spec')
        self.model.unit.status = MaintenanceStatus("Configuring pod")
        self.set_pod_spec()

        response = utils.create_pod_security_policy_with_api(
            namespace=self._stored.namespace,
        )
        if not response:
            self.model.unit.status = \
                BlockedStatus("An error occured during init. Please check the logs.")
            event.defer()
            return

$ juju model-config | grep logging
logging-config                     model       <root>=WARNING;unit=TRACE

application-metallb-speaker: 15:13:09 DEBUG unit.metallb-speaker/0.juju-log Operator Framework 0.9.0 up and running.
application-metallb-controller: 15:14:31 INFO juju.cmd.jujud caas operator application-metallb-controller start (2.8.3 [gc])
admissionregistration.k8s.io/v1beta1
apiextensions.k8s.io/v1
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1
    
  },
  "status": "Failure",
    "kind": "podsecuritypolicies"
  },
  "code": 403
}
application-metallb-controller: 15:14:45 ERROR unit.metallb-controller/22.juju-log Exception when calling PolicyV1beta1Api->create_pod_security_policy.
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-metallb-controller-22/charm/src/utils.py", line 57, in create_pod_security_policy_with_api
    api_instance.create_pod_security_policy(body, pretty=True)
    return self.api_client.call_api(
    response_data = self.request(
  File "/var/lib/juju/agents/unit-metallb-controller-22/charm/venv/kubernetes/client/api_client.py", line 382, in request
    return self.rest_client.POST(url,
  File "/var/lib/juju/agents/unit-metallb-controller-22/charm/venv/kubernetes/client/rest.py", line 272, in POST
Reason: Forbidden
  "kind": "Status",
    
  },
pe",
  "reason": "Forbidden",
  "details": {
    "group": "policy",
    "kind": "podsecuritypolicies"
  },
  "code": 403
}

application-metallb-controller: 15:14:46 DEBUG unit.metallb-controller/22.juju-log Operator Framework 0.9.0 up and running.
application-metallb-controller: 15:14:46 DEBUG unit.metallb-controller/22.juju-log Legacy hooks/config-changed does not exist.
application-metallb-controller: 15:14:46 DEBUG unit.metallb-controller/22.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-controller: 15:14:46 DEBUG unit.metallb-controller/22.juju-log Emitting Juju event config_changed.


application-metallb-speaker: 15:18:35 DEBUG unit.metallb-speaker/0.juju-log Operator Framework 0.9.0 up and running.
application-metallb-speaker: 15:18:35 DEBUG unit.metallb-speaker/0.juju-log Legacy hooks/update-status does not exist.
application-metallb-speaker: 15:18:35 DEBUG unit.metallb-speaker/0.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-speaker: 15:18:35 DEBUG unit.metallb-speaker/0.juju-log Emitting Juju event update_status.
application-metallb-controller: 15:20:32 DEBUG unit.metallb-controller/22.juju-log Operator Framework 0.9.0 up and running.
application-metallb-controller: 15:20:32 DEBUG unit.metallb-controller/22.juju-log Legacy hooks/update-status does not exist.
application-metallb-controller: 15:20:32 DEBUG unit.metallb-controller/22.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-controller: 15:20:32 DEBUG unit.metallb-controller/22.juju-log Emitting Juju event update_status.
application-metallb-speaker: 15:22:56 DEBUG unit.metallb-speaker/0.juju-log Operator Framework 0.9.0 up and running.
application-metallb-speaker: 15:22:56 DEBUG unit.metallb-speaker/0.juju-log Legacy hooks/update-status does not exist.
application-metallb-speaker: 15:22:56 DEBUG unit.metallb-speaker/0.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-speaker: 15:22:56 DEBUG unit.metallb-speaker/0.juju-log Emitting Juju event update_status.

application-metallb-controller: 15:26:20 DEBUG unit.metallb-controller/22.juju-log Operator Framework 0.9.0 up and running.
application-metallb-controller: 15:26:20 DEBUG unit.metallb-controller/22.juju-log Legacy hooks/update-status does not exist.
application-metallb-controller: 15:26:20 DEBUG unit.metallb-controller/22.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-controller: 15:26:20 DEBUG unit.metallb-controller/22.juju-log Emitting Juju event update_status.
application-metallb-speaker: 15:27:05 DEBUG unit.metallb-speaker/0.juju-log Operator Framework 0.9.0 up and running.
application-metallb-speaker: 15:27:05 DEBUG unit.metallb-speaker/0.juju-log Legacy hooks/update-status does not exist.
application-metallb-speaker: 15:27:05 DEBUG unit.metallb-speaker/0.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-speaker: 15:27:05 DEBUG unit.metallb-speaker/0.juju-log Emitting Juju event update_status.
application-metallb-controller: 15:30:56 DEBUG unit.metallb-controller/22.juju-log Operator Framework 0.9.0 up and running.
application-metallb-controller: 15:30:56 DEBUG unit.metallb-controller/22.juju-log Legacy hooks/update-status does not exist.
application-metallb-controller: 15:30:56 DEBUG unit.metallb-controller/22.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-controller: 15:30:56 DEBUG unit.metallb-controller/22.juju-log Emitting Juju event update_status.
application-metallb-speaker: 15:32:38 DEBUG unit.metallb-speaker/0.juju-log Operator Framework 0.9.0 up and running.
application-metallb-speaker: 15:32:38 DEBUG unit.metallb-speaker/0.juju-log Legacy hooks/update-status does not exist.
application-metallb-speaker: 15:32:38 DEBUG unit.metallb-speaker/0.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-speaker: 15:32:38 DEBUG unit.metallb-speaker/0.juju-log Emitting Juju event update_status.
application-metallb-controller: 15:35:41 DEBUG unit.metallb-controller/22.juju-log Operator Framework 0.9.0 up and running.
application-metallb-controller: 15:35:41 DEBUG unit.metallb-controller/22.juju-log Legacy hooks/update-status does not exist.
application-metallb-controller: 15:35:41 DEBUG unit.metallb-controller/22.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-controller: 15:35:41 DEBUG unit.metallb-controller/22.juju-log Emitting Juju event update_status.
application-metallb-speaker: 15:37:48 DEBUG unit.metallb-speaker/0.juju-log Operator Framework 0.9.0 up and running.
application-metallb-speaker: 15:37:48 DEBUG unit.metallb-speaker/0.juju-log Legacy hooks/update-status does not exist.
application-metallb-speaker: 15:37:48 DEBUG unit.metallb-speaker/0.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-speaker: 15:37:48 DEBUG unit.metallb-speaker/0.juju-log Emitting Juju event update_status.
application-metallb-controller: 15:40:07 DEBUG unit.metallb-controller/22.juju-log Operator Framework 0.9.0 up and running.
application-metallb-controller: 15:40:07 DEBUG unit.metallb-controller/22.juju-log Legacy hooks/update-status does not exist.
application-metallb-controller: 15:40:07 DEBUG unit.metallb-controller/22.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-controller: 15:40:07 DEBUG unit.metallb-controller/22.juju-log Emitting Juju event update_status.
application-metallb-speaker: 15:43:48 DEBUG unit.metallb-speaker/0.juju-log Operator Framework 0.9.0 up and running.
application-metallb-speaker: 15:43:48 DEBUG unit.metallb-speaker/0.juju-log Legacy hooks/update-status does not exist.
application-metallb-speaker: 15:43:48 DEBUG unit.metallb-speaker/0.juju-log Using controller storage: JUJU_VERSION=2.8.3
application-metallb-speaker: 15:43:48 DEBUG unit.metallb-speaker/0.juju-log Emitting Juju event update_status.

crodriguez@camille-ubuntu:~/Projects/metallb-bundle$ juju show-status-log metallb-controller/22
Time                        Type       Status       Message
18 Sep 2020 15:14:21-05:00  juju-unit  allocating   
18 Sep 2020 15:14:21-05:00  workload   waiting      agent initializing
18 Sep 2020 15:14:39-05:00  workload   maintenance  installing charm software
18 Sep 2020 15:14:39-05:00  juju-unit  executing    running install hook
18 Sep 2020 15:14:41-05:00  juju-unit  executing    running leader-elected hook
18 Sep 2020 15:14:42-05:00  juju-unit  executing    running config-changed hook
18 Sep 2020 15:14:44-05:00  juju-unit  executing    running start hook
18 Sep 2020 15:14:45-05:00  workload   maintenance  Configuring pod
18 Sep 2020 15:14:46-05:00  juju-unit  idle         
18 Sep 2020 15:14:46-05:00  juju-unit  executing    running config-changed hook
18 Sep 2020 15:14:47-05:00  juju-unit  allocating   Created container juju-pod-init
18 Sep 2020 15:14:47-05:00  juju-unit  allocating   Started container juju-pod-init
18 Sep 2020 15:14:52-05:00  juju-unit  idle         
18 Sep 2020 15:14:52-05:00  workload   blocked      An error occured during init. Please check the logs.

[camille-rodriguez]

Hi @chipaca , I’m sorry, which branch? I only see a “master” branch in this project. Did you mean to add a link?

Edit : ah nevermind, I checked too quickly, you mean “Log event deferral” branch from your repo. I will test soon and let you know!

[chipaca]

any news WRT testing the fix? I’d like confirmation before landing it…