Metricbeat not indexing data to Elasticsearch

Elastic Stack Beats
1

Hi,
I have got below JSON in metricbeat logs.

"@timestamp": "2018-06-01T04:00:33.851Z",
"@metadata": {
"beat": "metricbeat",
"type": "doc",
"version": "6.3.0"
},
"fields": {
"app_id": "aaa",
"type": "inframetric",
"appcode": "testAppCode",
"env": "uat",
"appclient": "xxx"
},
"totalMemory": 4156067840,
"percentMemUsed": 0.8415,
"totalSwap": 8316817408,
"beat": {
"name": "xxx",
"hostname": "xxx",
"version": "6.3.0"
},
"metricset": {
"name": "memory",
"module": "system"
},
"system": {
"memory": {
"free": 658735104,
"actual": {
"used": {
"bytes": 3497332736
},
"free": 658735104
},
"swap": {
"used": {
"bytes": 5700812800
},
"free": 2616004608
},
"used": {
"bytes": 3497332736,
"pct": 0.8415
}
}
},
"percentSwap": 0.6855,
"host": {
"name": "xxx"
}
}

Logstash output configuration:

if [@metadata][beat] == "metricbeat" {
elasticsearch{
hosts => ["ES-host:8300"]
index => "myIndex-metric-%{+YYYY.MM.dd}"
user => "elastic"
password => "elastic"
timeout => 300
}
else{
elasticsearch{
hosts => ["ES-host:8300"]
index => "myindex-syslog-%{+YYYY.MM.dd}"
user => "elastic"
password => "elastic"
timeout => 300

}

I am putting metricbeat fields on myIndex-metric diff index because second index has data coming form filebeat and its having same fields.
Is that the reason its now showing?
Should I put data from both metricbat and filebeat on same index?
fo example. myindex-syslog also has 'percentCpu' field and from metricbat configuration, I am renaming From:system.cpu.total.pct To:percentCpu
it shows the renaming is done in metricbat log, but doesnt show field under index myIndex-metric.
Why is that so?

2

It's better to keep Filebeat and Metricbeat separate.

If you use Logstash and change the index names, it's important to make sure that the FB and MB templates are adjusted accordingly and loaded in Elasticsearch. If you don't do that, the dashboards won't work. Please see this docs link: https://www.elastic.co/guide/en/apm/server/6.2/_manually_loading_template_configuration.html#load-template-manually

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.