Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Postgres logs filled with AURORA_VERSION() errors

See original GitHub issue

Output of the info page

===============
Agent (v6.22.0)
===============

  Status date: 2020-08-28 05:02:20.164462 UTC
  Agent start: 2020-08-27 17:58:39.626690 UTC
  Pid: 1107
  Go Version: go1.13.11
  Python Version: 2.7.18
  Build arch: amd64
  Agent flavor: agent
  Check Runners: 6
  Log Level: error

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    NTP offset: -983µs
    System UTC time: 2020-08-28 05:02:20.164462 UTC

  Host Info
  =========
    bootTime: 2020-05-01 23:55:20.000000 UTC
    kernelArch: x86_64
    kernelVersion: 5.4.0-0.bpo.2-cloud-amd64
    os: linux
    platform: debian
    platformFamily: debian
    platformVersion: 10.2
    procs: 587
    uptime: 2826h3m22s

  Hostnames
  =========
    <redacted>

  Metadata
  ========
    cloud_provider: AWS
    hostname_source: fqdn

=========
Collector
=========

  Running Checks
  ==============

    cpu
    ---
      Instance ID: cpu [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/cpu.d/conf.yaml.default
      Total Runs: 2,654
      Metric Samples: Last Run: 7, Total: 18,572
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2020-08-28 05:02:09.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:09.000000 UTC


    disk (2.10.1)
    -------------
      Instance ID: disk:2b0426306ced0e7e [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/disk.d/conf.yaml
      Total Runs: 2,655
      Metric Samples: Last Run: 40, Total: 106,200
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 4, Total: 10,620
      Average Execution Time : 41ms
      Last Execution Date : 2020-08-28 05:02:12.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:12.000000 UTC


    docker
    ------
      Instance ID: docker [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/docker.d/conf.yaml
      Total Runs: 2,655
      Metric Samples: Last Run: 934, Total: 2,491,463
      Events: Last Run: 0, Total: 445
      Service Checks: Last Run: 1, Total: 2,655
      Average Execution Time : 88ms
      Last Execution Date : 2020-08-28 05:02:19.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:19.000000 UTC


    file_handle
    -----------
      Instance ID: file_handle [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/file_handle.d/conf.yaml.default
      Total Runs: 2,655
      Metric Samples: Last Run: 5, Total: 13,275
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2020-08-28 05:02:16.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:16.000000 UTC


    http_check (4.10.0)
    -------------------
      <redacted>


    io
    --
      Instance ID: io [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/io.d/conf.yaml.default
      Total Runs: 2,654
      Metric Samples: Last Run: 52, Total: 137,972
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2020-08-28 05:02:08.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:08.000000 UTC


    load
    ----
      Instance ID: load [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/load.d/conf.yaml.default
      Total Runs: 2,655
      Metric Samples: Last Run: 6, Total: 15,930
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2020-08-28 05:02:15.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:15.000000 UTC


    memory
    ------
      Instance ID: memory [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/memory.d/conf.yaml.default
      Total Runs: 2,654
      Metric Samples: Last Run: 18, Total: 47,772
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2020-08-28 05:02:07.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:07.000000 UTC


    network (1.17.0)
    ----------------
      Instance ID: network:5c571333f400457d [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/network.d/conf.yaml.default
      Total Runs: 2,655
      Metric Samples: Last Run: 229, Total: 608,139
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 10ms
      Last Execution Date : 2020-08-28 05:02:14.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:14.000000 UTC


    nginx (3.8.1)
    -------------
      Instance ID: nginx:4cbf3ff6b6038603 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/nginx.d/conf.yaml
      Total Runs: 2,654
      Metric Samples: Last Run: 7, Total: 18,578
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 2,654
      Average Execution Time : 5ms
      Last Execution Date : 2020-08-28 05:02:11.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:11.000000 UTC


    ntp
    ---
      Instance ID: ntp:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/ntp.d/conf.yaml.default
      Total Runs: 45
      Metric Samples: Last Run: 1, Total: 45
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 45
      Average Execution Time : 1.02s
      Last Execution Date : 2020-08-28 04:58:42.000000 UTC
      Last Successful Execution Date : 2020-08-28 04:58:42.000000 UTC


    postgres (5.0.2)
    ----------------
      Instance ID: postgres:2ca9e58bf25aa4d8 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/postgres.d/conf.yaml
      Total Runs: 2,654
      Metric Samples: Last Run: 35, Total: 92,890
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 2,654
      Average Execution Time : 22ms
      Last Execution Date : 2020-08-28 05:02:05.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:05.000000 UTC
      metadata:
        version.major: 12
        version.minor: 1
        version.patch: 0
        version.raw: 12.1 (Debian 12.1-1.pgdg100+1)
        version.scheme: semver

      ...

    uptime
    ------
      Instance ID: uptime [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/uptime.d/conf.yaml.default
      Total Runs: 2,654
      Metric Samples: Last Run: 1, Total: 2,654
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2020-08-28 05:02:06.000000 UTC
      Last Successful Execution Date : 2020-08-28 05:02:06.000000 UTC

========
JMXFetch
========

  Initialized checks
  ==================
    <redacted>

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  Transactions
  ============
    CheckRunsV1: 2,654
    Connections: 0
    Containers: 0
    Deployments: 0
    Dropped: 0
    DroppedOnInput: 0
    Events: 0
    HostMetadata: 0
    IntakeV1: 598
    Metadata: 0
    Pods: 0
    Processes: 0
    RTContainers: 0
    RTProcesses: 0
    ReplicaSets: 0
    Requeued: 1
    Retried: 1
    RetryQueueSize: 0
    Series: 0
    ServiceChecks: 0
    Services: 0
    SketchSeries: 0
    Success: 5,906
    TimeseriesV1: 2,654

  Transaction Errors
  ==================
    Total number: 1
    Errors By Type:

  HTTP Errors
  ==================
    Total number: 1
    HTTP Errors By Code:
      504: 1

  API Keys status
  ===============
    API key ending with 8a1f5: API Key valid

==========
Endpoints
==========
  https://app.datadoghq.com - API Key ending with:
      - 8a1f5

==========
Logs Agent
==========

    Sending compressed logs in HTTPS to agent-http-intake.logs.datadoghq.com on port 443
    BytesSent: 2.05207004e+08
    EncodedBytesSent: 1.4129913e+07
    LogsProcessed: 148183
    LogsSent: 148168

  <redacted>

  journald
  --------
    Type: journald
    IncludeUnits: docker.service
    Status: OK
    Inputs: default

  <redacted>

=========
APM Agent
=========
  Status: Not running or unreachable on localhost:8126.
  Error: Get http://localhost:8126/debug/vars: dial tcp [::1]:8126: connect: connection refused

=========
Aggregator
=========
  Checks Metric Sample: 6,526,430
  Dogstatsd Metric Sample: 478,093
  Event: 446
  Events Flushed: 446
  Number Of Flushes: 2,654
  Series Flushed: 6,658,293
  Service Check: 154,543
  Service Checks Flushed: 157,159

=========
DogStatsD
=========
  Event Packets: 0
  Event Parse Errors: 0
  Metric Packets: 478,092
  Metric Parse Errors: 0
  Service Check Packets: 5,282
  Service Check Parse Errors: 0
  Udp Bytes: 56,492,000
  Udp Packet Reading Errors: 0
  Udp Packets: 80,820
  Uds Bytes: 0
  Uds Origin Detection Errors: 0
  Uds Packet Reading Errors: 0
  Uds Packets: 0

Additional environment details (Operating System, Cloud provider, etc):

We run most of our services and Postgres databases inside Docker containers on Debian Linux within AWS.

Steps to reproduce the issue:

  1. Setup a Postgres 12 Docker container
  2. Setup /etc/datadog-agent/conf.d/postgres.d/conf.yaml with a config that resembles the following to monitor this Postgres container:
init_config:

instances:
  - host: localhost
    port: <port>
    ssl: true
    username: <username>
    password: <password>
    dbname: <db_name>
    tags:
      ...

Describe the results you received: Using docker logs to inspect the database being monitored I see the following messages repeated every 15 seconds:

ERROR:  function aurora_version() does not exist at character 8
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.
STATEMENT:  select AURORA_VERSION();

Describe the results you expected: Datadog’s Aurora checks should not produce errors in the non-Aurora Postgres logs.

Additional information you deem important (e.g. issue happens only occasionally): These errors started happening when we upgraded our Datadog agent from 6.21.1 to 6.22.0. I believe this change introduced this regression: https://github.com/DataDog/integrations-core/pull/7123/files#diff-d069956e6d0c0319626e8f0682793aafR26-R35

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:6
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
zinosamacommented, Sep 1, 2020
0reactions
Sliwins0commented, Aug 11, 2021

We have same problem with our installation of Datadog integration (agent 7.29.1). PostgreSQL 12.7 on x86_64-pc-linux-gnu. image

Read more comments on GitHub >

github_iconTop Results From Across the Web

Working with RDS and Aurora PostgreSQL logs: Part 1
PostgreSQL generates event logs that contain useful information for DBAs. SQL query failures, failed login attempts, and deadlocks are captured ...
Read more >
20.8. Error Reporting and Logging - PostgreSQL
When logging to syslog is enabled, this parameter determines the program name used to identify PostgreSQL messages in syslog logs. The default is...
Read more >
AURORA_VERSION check on non aurora database logged ...
Symptom. Since Jira 8.4.0 having support for Aurora, Jira is causing errors on database logging side for AURORA_VERSION() check every hour.
Read more >
Amazon Aurora PostgreSQL updates
However, starting with the release of PostgreSQL versions 13.3, 12.8, 11.13, 10.18, and for all other later versions, Aurora version numbers aren't used....
Read more >
aws_rds_cluster | Resources | hashicorp/aws
To manage non-Aurora databases (e.g., MySQL, PostgreSQL, SQL Server, etc.) ... log types are supported: audit , error , general , slowquery ,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

(process.py:454) | No matching process '...' was found (when process is running fine)

Next page

Using unicode username/password for RabbitMQ agent will result in 401 unauthorised access