Promitor scraper fails to find resource mapping for Microsoft.Network

Report

The promitor scraper seems to be unable to fetch the NetworkInterface Metrics in Azure Monitor

Expected Behavior

Scraping NetworkInterface Metrics from Azure Monitor

Actual Behavior

Fatal log msgs in promitor scraper Failed to find resource mapping for provider: Microsoft.Network, resourceType: networkInterfaces

Steps to Reproduce the Problem

1. Configure NetworkInterface ResourceDiscoveryGroup in promitor discovery agent
2. Scrape Metrics from NetworkInterface ind promitor agent scraper through ResourceDiscoveryGrooup

Component

Scraper

Version

2.7.3

Configuration

Configuration:

---
repositories:
  - name: promitor
    url: https://charts.promitor.io/
  - name: bedag
    url: https://bedag.github.io/helm-charts

releases:
  - name: promitor-scraper
    namespace: promitor
    chart: promitor/promitor-agent-scraper
    values:
      - azureAuthentication:
          mode: UserAssignedManagedIdentity
          identity:
            id: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_scraper_client_id") }}
            binding: promitor-scraper-identity
      - azureMetadata:
          tenantId: {{ requiredEnv "ARM_TENANT_ID" }}
          subscriptionId: {{ requiredEnv "ARM_SUBSCRIPTION_ID" }}
          resourceGroupName: REDACTED
      - resourceDiscovery:
          enabled: true
          host: promitor-agent-resource-discovery
          port: 8889
      - metricSinks:
          prometheusScrapingEndpoint:
            enabled: true
            serviceMonitor:
              enabled : true
              namespace: promitor
          enableServiceDiscovery: true
      - metrics:
          - name: azure_network_interface_bytes_received_rate
            description: "Number of bytes the Network Interface received."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: BytesReceivedRate
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: network-interfaces-landscape
          - name: azure_network_interface_bytes_sent_rate
            description: "Number of bytes the Network Interface sent."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: BytesSentRate
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: network-interfaces-landscape
          - name: azure_network_interface_packets_received_rate
            description: "Number of packets the Network Interface received."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: PacketsReceivedRate
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: network-interfaces-landscape
          - name: azure_network_interface_packets_sent_rate
            description: "Number of packets the Network Interface sent."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: PacketsSentRate
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: network-interfaces-landscape
      - tolerations:
          - key: "CriticalAddonsOnly"
            operator: "Exists"
            effect: "NoSchedule"
  - name: promitor-discovery
    namespace: promitor
    chart: promitor/promitor-agent-resource-discovery
    values:
      - azureAuthentication:
          mode: UserAssignedManagedIdentity
          identity:
            id: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_discovery_client_id") }}
            binding: promitor-discovery-identity
      - azureLandscape:
          cloud: Global
          tenantId: {{ requiredEnv "ARM_TENANT_ID" }}
          subscriptions:
            - {{ requiredEnv "ARM_SUBSCRIPTION_ID" }}
      - prometheus:
          serviceMonitor:
            enabled: true
            namespace: promitor
      - resourceDiscoveryGroups:
          - name: network-interfaces-landscape
            type: NetworkInterface
      - tolerations:
          - key: "CriticalAddonsOnly"
            operator: "Exists"
            effect: "NoSchedule"
  - name: promitor-scraper-identity
    namespace: promitor
    chart: ../../charts/azure-identity
    values:
      - identityName: promitor-scraper-identity
      - resourceId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_scraper_resource_id") }}
      - clientId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_scraper_client_id") }}
      - selector: promitor-scraper-identity
  - name: promitor-discovery-identity
    namespace: promitor
    chart: ../../charts/azure-identity
    values:
      - identityName: promitor-discovery-identity
      - resourceId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_discovery_resource_id") }}
      - clientId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_discovery_client_id") }}
      - selector: promitor-discovery-identity

Logs

2022-04-28T08:10:05.445290049Z stdout F [08:10:05 FTL] NotFound: Failed to find resource mapping for provider: Microsoft.Network, resourceType: networkInterfaces, id: /subscriptions/{SUBSCRIPTIONID}/resourceGroups/{RESOURCEGROUP}/providers/Microsoft.Network/networkInterfaces/{PODNAME}.nic.ee13ddf1-772a-4855-822a-a5ba2be2a9ea, traceId: {600df2dd-b4fb-46f5-85b3-f241561f791c}
2022-04-28T08:10:04.545832034Z stdout F [08:10:04 FTL] NotFound: Failed to find resource mapping for provider: Microsoft.Network, resourceType: networkInterfaces, id: /subscriptions/{SUBSCRIPTIONID}/resourceGroups/{RESOURCEGROUP}/providers/Microsoft.Network/networkInterfaces/{PODNAME}.nic.c88992f6-5ecd-44d1-b332-84795d619807, traceId: {be611765-5fbd-4b71-aafd-01e0a4e814e4}
2022-04-28T08:10:03.869797215Z stdout F [08:10:03 FTL] NotFound: Failed to find resource mapping for provider: Microsoft.Network, resourceType: networkInterfaces, id: /subscriptions/{SUBSCRIPTIONID}/resourceGroups/{RESOURCEGROUP}/providers/Microsoft.Network/networkInterfaces/{PODNAME}.nic.327f47ff-6dae-4bd9-9cca-729917de5d40, traceId: {8f662120-c90c-478e-bc4b-b6b7f9bf6cf5}
2022-04-28T08:10:03.838297078Z stdout F [08:10:03 FTL] NotFound: Failed to find resource mapping for provider: Microsoft.Network, resourceType: networkInterfaces, id: /subscriptions/{SUBSCRIPTIONID}/resourceGroups/{RESOURCEGROUP}/providers/Microsoft.Network/networkInterfaces/{PODNAME}.nic.81f30dfd-cfeb-46d9-aae9-5cb99f7355bd, traceId: {726b500d-7b77-42f2-90c3-4e49deae33d2}
2022-04-28T08:10:03.740632441Z stdout F [08:10:03 FTL] NotFound: Failed to find resource mapping for provider: Microsoft.Network, resourceType: networkInterfaces, id: /subscriptions/{SUBSCRIPTIONID}/resourceGroups/{RESOURCEGROUP}/providers/Microsoft.Network/networkInterfaces/{PODNAME}.nic.c88992f6-5ecd-44d1-b332-84795d619807, traceId: {de2b765a-bbe8-4575-b02b-bcfe05f55a81}
2022-04-28T08:10:03.612416274Z stdout F [08:10:03 FTL] NotFound: Failed to find resource mapping for provider: Microsoft.Network, resourceType: networkInterfaces, id: /subscriptions/{SUBSCRIPTIONID}/resourceGroups/{RESOURCEGROUP}/providers/Microsoft.Network/networkInterfaces/{PODNAME}.nic.ac66fc98-779a-421e-a4fe-98736cc3d489, traceId: {8c173ef7-156e-46c2-b617-322f5b107361}

Platform

Microsoft Azure

Contact Details

No response

[tomkerkhove]

Hm that looks pretty straightforward. I’ll need to try to reproduce this, sorry.

[amirschw]

I wonder if migrating to the new Azure SDKs might help - https://github.com/tomkerkhove/promitor/issues/2209