Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

traefik errors on cos layer with error: hook failed: "metrics-endpoint-relation-joined"

See original GitHub issue

Bug Description

Solutions QA team has two runs in which traefik errors on cos layer with error: hook failed: “metrics-endpoint-relation-joined”

The cos layer is built on top of kubernetes-aws.

From the logs:

2022-08-21 02:20:42 ERROR juju-log metrics-endpoint:10: Uncaught exception while in charm code: Traceback (most recent call last): File “./src/charm.py”, line 678, in <module> main(TraefikIngressCharm) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/main.py”, line 431, in main _emit_charm_event(charm, dispatcher.event_name) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/main.py”, line 142, in _emit_charm_event event_to_emit.emit(*args, **kwargs) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/framework.py”, line 316, in emit framework._emit(event) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/framework.py”, line 784, in _emit self._reemit(event_path) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/framework.py”, line 857, in _reemit custom_handler(event) File “/var/lib/juju/agents/unit-traefik-0/charm/lib/charms/prometheus_k8s/v0/prometheus_scrape.py”, line 1545, in _set_scrape_job_spec self._set_unit_ip(event) File “/var/lib/juju/agents/unit-traefik-0/charm/lib/charms/prometheus_k8s/v0/prometheus_scrape.py”, line 1576, in _set_unit_ip unit_ip = str(self._charm.model.get_binding(relation).network.bind_address) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/model.py”, line 679, in network self._network = self._network_get(self.name, self._relation_id) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/model.py”, line 672, in _network_get return Network(self._backend.network_get(name, relation_id)) File “/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/model.py”, line 724, in init self.ingress_addresses.append(ipaddress.ip_address(address)) File “/usr/lib/python3.8/ipaddress.py”, line 53, in ip_address raise ValueError(‘%r does not appear to be an IPv4 or IPv6 address’ % ValueError: ‘acb396cf5563d429f9ebc8aa23ae47ed-1516332866.us-east-1.elb.amazonaws.com’ does not appear to be an IPv4 or IPv6 address

Failed runs:

https://solutions.qa.canonical.com/testruns/testRun/0b35ee4c-efcf-4f50-a393-696803b31ac9 https://solutions.qa.canonical.com/testruns/testRun/ef230601-4af3-4d6f-89af-4e578985e666

Logs are found on the bottom on the page, on the artifacts repository.

To Reproduce

These errors were found on the fkb-master-kubernetes-focal-aws SKU of our automated test suite.

Environment

charmed-kubernetes 1.24 stable channel for cos charms.

Relevant log output

Traceback (most recent call last):
  File "./src/charm.py", line 678, in <module>
    main(TraefikIngressCharm)
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/main.py", line 431, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/main.py", line 142, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/framework.py", line 316, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/framework.py", line 784, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/framework.py", line 857, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-traefik-0/charm/lib/charms/prometheus_k8s/v0/prometheus_scrape.py", line 1545, in _set_scrape_job_spec
    self._set_unit_ip(event)
  File "/var/lib/juju/agents/unit-traefik-0/charm/lib/charms/prometheus_k8s/v0/prometheus_scrape.py", line 1576, in _set_unit_ip
    unit_ip = str(self._charm.model.get_binding(relation).network.bind_address)
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/model.py", line 679, in network
    self._network = self._network_get(self.name, self._relation_id)
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/model.py", line 672, in _network_get
    return Network(self._backend.network_get(name, relation_id))
  File "/var/lib/juju/agents/unit-traefik-0/charm/venv/ops/model.py", line 724, in __init__
    self.ingress_addresses.append(ipaddress.ip_address(address))
  File "/usr/lib/python3.8/ipaddress.py", line 53, in ip_address
    raise ValueError('%r does not appear to be an IPv4 or IPv6 address' %
ValueError: 'acb396cf5563d429f9ebc8aa23ae47ed-1516332866.us-east-1.elb.amazonaws.com' does not appear to be an IPv4 or IPv6 address