How to investigate "Could not create module" errors?

Expected Behavior

Module starts. If module does not start because of an error should be shown, which indicates what has really caused the module to fail. The only error I see is:

Container runtime error (See below pasted log)

This is not helpful. Unfortunately, because the module does not start, it is also not possible to see any log information.

QUESTION: What is the best practice to investigate such kind of issue?

Current Behavior

Module cannot be loaded, because of an error. It is probably some error in module itself or some configuration error. This is exactly the issue, because it is not know what kind of error it is.

Steps to Reproduce

Deploy module

Context (Environment)

Output of iotedge check

Click here

Configuration checks
--------------------
√ config.yaml is well-formed - OK
√ config.yaml has well-formed connection string - OK
√ container engine is installed and functional - OK
‼ config.yaml has correct hostname - Warning
    config.yaml has hostname Abcde-SR1 which does not comply with RFC 1035.

    - Hostname must be between 1 and 255 octets inclusive.
    - Each label in the hostname (component separated by ".") must be between 1 and 63 octets inclusive.
    - Each label must start with an ASCII alphabet character (a-z), end with an ASCII alphanumeric character (a-z, 0-9), and must contain only ASCII alphanumeric characters or hyphens (a-z, 0-9, "-").

    Not complying with RFC 1035 may cause errors during the TLS handshake with modules and downstream devices.
√ config.yaml has correct URIs for daemon mgmt endpoint - OK
√ latest security daemon - OK
√ host time is close to real time - OK
√ container time is close to host time - OK
‼ DNS server - Warning
    Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
    Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
    You can ignore this warning if you are setting DNS server per module in the Edge deployment.
‼ production readiness: certificates - Warning
    Device is using self-signed, automatically generated certs.
    Please see https://aka.ms/iotedge-prod-checklist-certs for best practices.
√ production readiness: certificates expiry - OK
‼ production readiness: container engine - Warning
    Device is not using a production-supported container engine (moby-engine).
    Please see https://aka.ms/iotedge-prod-checklist-moby for details.
‼ production readiness: logs policy - Warning
    Container engine is not configured to rotate module logs which may cause it run out of disk space.
    Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
    You can ignore this warning if you are setting log policy per module in the Edge deployment.

Connectivity checks
-------------------
√ host can connect to and perform TLS handshake with IoT Hub AMQP port - OK
√ host can connect to and perform TLS handshake with IoT Hub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with IoT Hub MQTT port - OK
√ container on the IoT Edge module network can connect to IoT Hub AMQP port - OK
√ container on the IoT Edge module network can connect to IoT Hub HTTPS / WebSockets port - OK
√ container on the IoT Edge module network can connect to IoT Hub MQTT port - OK
√ Edge Hub can bind to ports on host - OK

15 check(s) succeeded.

Device (Host) Operating System

Windows 10

Architecture

amd64,

Container Operating System

Linux containers

Runtime Versions

iotedged

iotedge 1.0.8

Edge Agent

1.0.8

Edge Hub

1.0.8

Docker

Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:26:49 2019
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          3.0.5
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.12.1
  Git commit:       dbe4a30
  Built:            Thu Apr 18 22:07:58 2019
  OS/Arch:          windows/amd64
  Experimental:     false

Logs

edge-agent logs

2019-09-24 07:45:18.506 +00:00 [WRN] - Reconcile failed because of the an exception
System.AggregateException: One or more errors occurred. (Error calling Create module publisher: Could not create module publisher
        caused by: Could not create module publisher
        caused by: Container runtime error
        caused by: expected value at line 1 column 1) ---> Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException: Error calling Create module publisher: Could not create module publisher
        caused by: Could not create module publisher
        caused by: Container runtime error
        caused by: expected value at line 1 column 1

[CindyXing]

Besides letting customers to look at logs at edge, I think we should upstream any error/failure to the cloud. Potentially the telemetry channel can be used.

[ddobric]

This is solved. Please take a look here.

Thanks for support.