Free memory metrics don't include buff/cache

Describe the bug The Java APM agent fails to capture accurate memory and CPU metrics in a containerised env. The cpu and memory statistics for the underlying host are reported NOT the container. This manifests itself in a k8 env but is probably also an issue in a simple docker deployment.

To Reproduce

1. Deploy an app running in a k8 env - the opbeans example is sufficient. k8 configuration can be provided if needed.
2. Check metrics tab of agent

As illustrated above, memory is always almost 100%. The docs suggest the values of the underlying host are used - in this case 30GB.

{
          "system" : {
            "memory" : {
              "actual" : {
                "free" : 8.82143232E8
              },
              "total" : 3.1629697024E10
            }
          }
        }

dalemcdiarmid@gke-eden-test-default-pool-18a6be9a-8qkm ~ $ free
              total        used        free      shared  buff/cache   available
Mem:       30888376     5974440     1329996        3956    23583940    24656560

Expected behaviour

CPU and memory should represent those of the pod with respect to their limits. Similar to how metricbeat collects.

Debug logs NA

The approach of each agent seems to be different here - the node agent uses cgroups for example.

[felixbarny]

It has no transitive dependencies.

Correction: it does have a transitive dependency on JNA (not JNI) and JNA can’t be shaded: https://github.com/java-native-access/jna/issues/679. Bummer! It would also add > 3MB to the agent jar file.

I think I’ll start with reading /proc/meminfo for Linux and use OperatingSystemMXBean#getFreePhysicalMemorySize otherwise.

[axw]

The Windows implementation of OperatingSystemMXBean.getFreePhysicalMemorySize uses GlobalMemoryStatusEx and returns the ullAvailPhys field of MEMORYSTATUSEX. Similarly, OperatingSystemMXBean.html#getTotalPhysicalMemorySize uses the ullTotalPhys field. Code: https://github.com/openjdk/jdk/blob/master/src/jdk.management/windows/native/libmanagement_ext/OperatingSystemImpl.c#L136-L154.

These are the same values that Metricbeat and the Go agent use. If JNI/JNA is a pain (which it probably is), then you could use OperatingSystemMXBean on Windows.

For macOS, I don’t think it really matters too much if we’re a little inconsistent, which it will be if you use OperatingSystemMXBean there.