edgeHub and modules are not able to connect with AMQP or MQTT using modem connection

Behavior

I have an Ubuntu 20.04 device with integrated modem/LTE device. When connecting with LAN cable everything works perfect, but when only when using the modem, the edgeHub and other modules fail to connect to IoTHub

Current Behavior

On LAN cable, all protocols are OK when doing ‘iotedge check’ As you can see in the ‘iotedge check’ below on modem connection only (LAN disconnected), the host is able to connect to iothub using all protocols. But on modem connection only, the default container network and azure-iot-edge network fail to connect to the Websockets ports. Reason is unknown. But they do allow connection on both AMQP and MQTT ports. I forced both edgeAgent and edgeHub to connection via MQTT (UpStreamProtocol = MQTT), and added a startupsequence for the modules. Only the edgeAgent connects, edgeHub and other modules fail to connect.

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

1. Only modem connection to internet
2. Set env “UpStreamProtocol” = “MQTT”
3. Check logs of edgeHub

Output of iotedge check

Click here

[iotedge-check.txt](https://github.com/Azure/iotedge/files/10013930/iotedge-check.txt)

Device Information

• Host OS: Ubuntu 20.04
• Architecture: amd64
• Container OS: Linux containers

Runtime Versions

• aziot-edged: 1.4.2
• Edge Agent: 1.4
• Edge Hub: 1.4
• Docker/Moby: 20.10.20+azure-1

Logs

aziot-edged logs

[aziot-edged1.txt](https://github.com/Azure/iotedge/files/10013936/aziot-edged1.txt)

edgeAgent Log

[edgeAgent_log1.txt](https://github.com/Azure/iotedge/files/10013940/edgeAgent_log1.txt)

edge-hub logs

[edgeHub_log1.txt](https://github.com/Azure/iotedge/files/10013948/edgeHub_log1.txt)

docker inspect network bridge

[bridge.json.txt](https://github.com/Azure/iotedge/files/10013952/bridge.json.txt)

docker inspect network azure-iot-edge

[azure-iot-edge.json.txt](https://github.com/Azure/iotedge/files/10013954/azure-iot-edge.json.txt)

docker inspect network host

[host.json.txt](https://github.com/Azure/iotedge/files/10013964/host.json.txt)

docker inspect edgeAgent

[edgeAgent.json.txt](https://github.com/Azure/iotedge/files/10013969/edgeAgent.json.txt)

docker inspect edgeHub

[edgeHub.json.txt](https://github.com/Azure/iotedge/files/10013976/edgeHub.json.txt)

[wvangeem]

Hi @nyanzebra, Just to inform you that this issue is solved now. I had some new insights in the problem, and the information is in following issue: #6928

[wvangeem]

@nyanzebra

I executed openssl s_client -connect iothub:8883 I hope it’s the command you suggested. I added the printed data below. This was with modem only: openssl.host.mqtt.txt As iotedge check points out, it is never a problem on the host. I tried it inside the edgeHub container, but it is missing the openssl command.

Here is a support-bundle when connected with LAN support_bundle_2022_11_23_14_55_10_UTC.zip

I connected LAN at 2022-11-23 14:53:46. You can see this in the edgeHub_log.txt file, where suddenly the connection resolved and everything starts to work fine.

I disconnected LAN at 2022-11-23 15:12 and added the support-bundle below. You can see in the edgeHub_log.txt that things go wrong again, whereas I can still ping to google.com from within the edgeHub container. I must say that generating a support bundle with modem takes a few minutes because of all the timeouts. With LAN it takes a few seconds. support_bundle_2022_11_23_15_13_58_UTC.zip