Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Null values for metrics if interval is set to 1 minute

See original GitHub issue

If we are setting the interval to 1 minute we see often (>90% of the cases) null metric values. I then started to play a bit with the azure metrics api and I think I have found out why.

// 5 minutes interval GET: https://management.azure.com/subscriptions/SID/resourceGroups/RG/providers/Microsoft.Compute/virtualMachines/VMname/providers/microsoft.insights/metrics?api-version=2018-01-01&interval=PT5M -> is fine

// 1 minute interval GET: https://management.azure.com/subscriptions/SID/resourceGroups/RG/providers/Microsoft.Compute/virtualMachines/VMname/providers/microsoft.insights/metrics?api-version=2018-01-01&interval=PT1M Body:

{
    "cost": 0,
    "timespan": "2020-09-22T11:40:00Z/2020-09-22T12:40:00Z",
    "interval": "PT1M",
    "value": [
        {
            "id": "...",
            "type": "Microsoft.Insights/metrics",
            "name": {
                "value": "Percentage CPU",
                "localizedValue": "Percentage CPU"
            },
            "displayDescription": "The percentage of allocated compute units that are currently in use by the Virtual Machine(s)",
            "unit": "Percent",
            "timeseries": [
                {
                    "metadatavalues": [],
                    "data": [
                        ...
                        {
                            "timeStamp": "2020-09-22T12:36:00Z",
                            "average": 4.285
                        },
                        {
                            "timeStamp": "2020-09-22T12:37:00Z",
                            "average": 3.4
                        },
                        {
                            "timeStamp": "2020-09-22T12:38:00Z"
                        },
                        {
                            "timeStamp": "2020-09-22T12:39:00Z"
                        }
                    ]
                }
            ],
            "errorCode": "Success"
        }
    ],
    "namespace": "Microsoft.Compute/virtualMachines",
    "resourceregion": "westeurope"
}

As you see the most recent entries don’t contain the average attribute. Sometimes the last 2 are missing sometimes only the most recent one. Pretty rare it’s fine.

I’m not at all familiar with azure metrics api. Do you know if this this the usual behavior of azure which should be covered in Promitor or is it more related to an azure issue?

Expected Behavior

Found value 3.4 for metric azure_virtual_machine_percentage_cpu with aggregation interval 00:01:00

Actual Behavior

Found value null for metric azure_virtual_machine_percentage_cpu with aggregation interval 00:01:00

Steps to Reproduce the Problem

Start scraper with the scrap metric definition from below or just fire the requests from above an a virtual machine resource

Configuration

Provide insights in the configuration that you are using:

Configured scraping schedule:

Used scraping configuration

- name: azure_virtual_machine_percentage_cpu
    description: "Average percentage cpu usage on an Azure virtual machine"
    resourceType: VirtualMachine
    scraping:
      schedule: "0 */2 * ? * *"
    azureMetricConfiguration:
      metricName: Percentage CPU
      aggregation:
        type: Average
        interval: 00:01:00
    resourceDiscoveryGroups:
    - name: vm-landscape