Error: container on the default network can connect to upstream HTTPS / WebSockets port

Expected Behavior

Create Edge Agent and Edge Hub when the device is child device (low layer device).

Current Behavior

Runtime is not working due to the “container on the default network can connect to upstream HTTPS / WebSockets port” is failed. TLS (HTTPS, MQTT, AMQP) connections success.
Please check my Output of iotedge check logs.

In device’s config.toml, I set parent_hostname as gateway machine’s hostname. When I set hostname as this device machine’s hostname instead, it works well. However, in this way, the device machine is connected to IoT Hub directly. I want to make the device is connected to gateway device, instead IoT Hub.

Also, without changing any settings, it connects once out of 100 times. this is weird…

Also, only upstream HTTPS/Websocket port is not working… checking MQTT and AMQP for container networking is passed. Please check my Output of iotedge check logs.

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

1. I have changed hostname IP to FQDN
2. I have checked that the image is pulled from registry
3. I have checked the TLS connection between gateway and device.

Context (Environment)

Output of iotedge check

Click here

Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
√ host time is close to reference time - OK
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
√ read all preloaded certificates from the Certificates Service - OK
√ read all preloaded key pairs from the Keys Service - OK
√ check all EST server URLs utilize HTTPS - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK

Connectivity checks (aziot-identity-service)
--------------------------------------------
√ host can connect to and perform TLS handshake with iothub AMQP port - OK
√ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with iothub MQTT port - OK

Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
√ configuration up-to-date with config.toml - OK
√ container engine is installed and functional - OK
√ configuration has correct parent_hostname - OK
√ parent hostname is resolvable from inside container - OK
× configuration has correct URIs for daemon mgmt endpoint - Error
    SocketError - SocketErrorCode (TimedOut) : Operation timed out
    One or more errors occurred. (Got bad response: )
        caused by: SocketError - SocketErrorCode (TimedOut) : Operation timed out
                   One or more errors occurred. (Got bad response: )
        caused by: docker returned exit status: 1, stderr = SocketError - SocketErrorCode (TimedOut) : Operation timed out
                   One or more errors occurred. (Got bad response: )
√ container time is close to host time - OK
‼ DNS server - Warning
    Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
    Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
    You can ignore this warning if you are setting DNS server per module in the Edge deployment.
        caused by: Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
                   Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
                   You can ignore this warning if you are setting DNS server per module in the Edge deployment.
‼ production readiness: logs policy - Warning
    Container engine is not configured to rotate module logs which may cause it run out of disk space.
    Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
    You can ignore this warning if you are setting log policy per module in the Edge deployment.
        caused by: Container engine is not configured to rotate module logs which may cause it run out of disk space.
                   Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
                   You can ignore this warning if you are setting log policy per module in the Edge deployment.
‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning
    The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
        caused by: The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
                   Data might be lost if the module is deleted or updated.
                   Please see https://aka.ms/iotedge-storage-host for best practices.
× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error
    Could not check current state of edgeHub container
        caused by: Could not check current state of edgeHub container
        caused by: docker returned exit status: 1, stderr = Error: No such object: edgeHub
√ Agent image is valid and can be pulled from upstream - OK
√ proxy settings are consistent in aziot-edged, aziot-identityd, moby daemon and config.toml - OK

Connectivity checks
-------------------
√ container on the default network can connect to upstream AMQP port - OK
 HTTPS / WebSockets port× container on the default network can connect to upstream HTTPS / WebSockets port - Error
    Container on the default network could not connect to yyh-gateway.southeastasia.cloudapp.azure.com:443
        caused by: Container on the default network could not connect to yyh-gateway.southeastasia.cloudapp.azure.com:443
        caused by: docker returned exit status: 1, stderr = One or more errors occurred. (A task was canceled.)
√ container on the default network can connect to upstream MQTT port - OK
    skipping because of not required in this configuration
√ container on the IoT Edge module network can connect to upstream AMQP port - OK
× container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error
    Container on the azure-iot-edge network could not connect to yyh-gateway.southeastasia.cloudapp.azure.com:443
        caused by: Container on the azure-iot-edge network could not connect to yyh-gateway.southeastasia.cloudapp.azure.com:443
        caused by: docker returned exit status: 1, stderr = One or more errors occurred. (A task was canceled.)
√ container on the IoT Edge module network can connect to upstream MQTT port - OK
    skipping because of not required in this configuration
28 check(s) succeeded.
3 check(s) raised warnings.
4 check(s) raised errors.
2 check(s) were skipped due to errors from other checks.

Device Information

*Host OS: Ubuntu 20.04

• Architecture: arm64
• Container OS : Linux containers

Runtime Versions

• aziot-edged [run iotedge version]: 1.4.2
• Edge Agent [image tag (e.g. 1.0.0)]: 1.4
• Edge Hub [image tag (e.g. 1.0.0)]: 1.4
• Docker/Moby [run docker version]: 20.10.21+azure-1

Note: when using Windows containers on Windows, run docker -H npipe:////./pipe/iotedge_moby_engine version instead

Logs

aziot-edged logs

<Paste here between the triple backticks>

edge-agent logs

<Paste here between the triple backticks>

edge-hub logs

<Paste here between the triple backticks>

Additional Information

There is no edge-agent logs, edge-hub logs…

$sudo iotedge list

A module runtime error occurred caused by: A module runtime error occurred caused by: timed out

[nyanzebra]

@varunpuranik tagging,

Reassigning to @and-rewsmith

[yeonhooy]

@and-rewsmith Appreciate for checking my problem. In my progress, I just set my environment by starting child’s runtime first. But, it does not make any other problem. So, I will do my work in here. If I struggle with another problem, I will open support ticket. Thanks!