Metricbeat not collecting memory info

[francopp (Franco Papalardo)]

After some Swap configuration changes metricbeat stop sending memory info I got any other type but memory info.

About an hour ago we made some changes in our server configuration and now we got this

The changes we made were to just stop using swap memory:

swapoff --all

Then change the fstab file and comment out the swap lines and finally a cache clear:

sync; echo 1 > /proc/sys/vm/drop_caches

sync; echo 2 > /proc/sys/vm/drop_caches

sync; echo 3 > /proc/sys/vm/drop_caches

After this we stop recieving memory part. And before this everything was normal. When I start metricbeat it's sending memory info in JSON but all data comming from memory it's all 0s

[andrewkroh]

Are there any errors in the log file?

Are there any error events reported to elasticsearch? You can query for _exists_:error on the Discover page in Kibana.

[francopp (Franco Papalardo)]

No errors in the log. Data is coming like this

{
  "@timestamp": "2017-03-27T14:42:08.408Z",
  "beat": {
    "hostname": "srvsig23logprod",
    "name": "srvsig23logprod",
    "version": "5.1.2"
  },
  "metricset": {
    "module": "system",
    "name": "process",
    "rtt": 187062
  },
  "system": {
    "process": {
      "cpu": {
        "start_time": "2016-05-30T16:36:15.000Z",
        "total": {
          "pct": 0
        }
      },
      "memory": {
        "rss": {
          "bytes": 0,
          "pct": 0
        },
        "share": 0,
        "size": 0
      },
      "name": "ksoftirqd/0",
      "pgid": 0,
      "pid": 4,
      "ppid": 2,
      "state": "sleeping",
      "username": "root"
    }
  },
  "type": "metricsets"
}

[andrewkroh]

You are showing an event from the process metricset (which looks normal BTW). Can you look for events from the memory metricset (metricset.name:memory). Are there any events from this metricset that have an error (metricset.name:memory AND _exists_:error)?

The memory data is read from /proc/meminfo on Linux. What's in that file?

[francopp (Franco Papalardo)]

Yeah, It was just an example of how logs are coming, as normal as always.

No output from the search, also try to filter by metricset.name and look up for memory but no logs since this changes.

This is my output of:

-bash-4.1$ cat /proc/meminfo
MemTotal: 132028084 kB
MemFree: 84576776 kB
Buffers: 183948 kB
Cached: 8654160 kB
SwapCached: 0 kB
Active: 7526564 kB
Inactive: 3658672 kB
Active(anon): 2437912 kB
Inactive(anon): 29164 kB
Active(file): 5088652 kB
Inactive(file): 3629508 kB
Unevictable: 34746100 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 4 kB
Dirty: 18980 kB
Writeback: 0 kB
AnonPages: 37093272 kB
Mapped: 181600 kB
Shmem: 272 kB
Slab: 583976 kB
SReclaimable: 469328 kB
SUnreclaim: 114648 kB
KernelStack: 19968 kB
PageTables: 88848 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 66014040 kB
Committed_AS: 38544732 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 750196 kB
VmallocChunk: 34289449660 kB
HardwareCorrupted: 0 kB
AnonHugePages: 36560896 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 12288 kB
DirectMap2M: 2011136 kB
DirectMap1G: 132120576 kB

[francopp (Franco Papalardo)]

There is no information for any problem like this before. Any idea?