Greetings,
I have been tasked by my team to extract status of our machines using Metricbeat. However I have observed Metricbeat emits message by each metric basis:
{"@metadata":{"type":"metricsets","beat":"metricbeat"},"system":{"process":{"pid":14016,"name":"gvim.exe","cmdline":"\"C:\\Program Files (x86)\\Vim\\vim80\\gvim.exe\" \"C:\\Users\\ext_tienjingw\\Desktop\\metricbeat-5.5.1-windows-x86_64\\metricbeat.full.yml\"","pgid":0,"memory":{"size":5550080,"rss":{"bytes":16551936,"pct":0.001},"share":0},"username":"TREND\\ext_tienjingw","state":"running","cpu":{"total":{"pct":0.0},"start_time":"2017-07-28T06:43:05.027Z"},"ppid":4624}},"metricset":{"module":"system","name":"process","rtt":120466},"beat":{"name":"tw-tienjingwang","hostname":"tw-tienjingwang","version":"5.5.1"},"@timestamp":"2017-07-28T06:51:06.919Z","type":"metricsets"}
{"@metadata":{"type":"metricsets","beat":"metricbeat"},"beat":{"version":"5.5.1","name":"tw-tienjingwang","hostname":"tw-tienjingwang"},"@timestamp":"2017-07-28T06:51:06.919Z","type":"metricsets","system":{"process":{"pid":8692,"ppid":7040,"username":"TREND\\ext_tienjingw","cmdline":"\"C:\\Program Files (x86)\\Vim\\vim80\\gvim.exe\" \"C:\\Users\\ext_tienjingw\\Desktop\\metricbeat-5.5.1-windows-x86_64\\logs\\metricbeat\"","cpu":{"total":{"pct":0.0},"start_time":"2017-07-28T06:48:56.172Z"},"name":"gvim.exe","state":"running","memory":{"share":0,"size":5390336,"rss":{"bytes":16453632,"pct":0.001}},"pgid":0}},"metricset":{"name":"process","rtt":120466,"module":"system"}}
{"@metadata":{"type":"metricsets","beat":"metricbeat"},"system":{"process":{"cpu":{"total":{"pct":0.0},"start_time":"2017-07-28T06:49:10.396Z"},"pgid":0,"state":"running","name":"cmd.exe","memory":{"rss":{"bytes":2895872,"pct":0.0002},"share":0,"size":1667072},"ppid":4624,"username":"TREND\\ext_tienjingw","cmdline":"\"cmd.exe\" /s /k pushd \"C:\\Users\\ext_tienjingw\\Desktop\\metricbeat-5.5.1-windows-x86_64\"","pid":12500}},"metricset":{"module":"system","name":"process","rtt":120466},"beat":{"name":"tw-tienjingwang","hostname":"tw-tienjingwang","version":"5.5.1"},"@timestamp":"2017-07-28T06:51:06.919Z","type":"metricsets"}
{"@metadata":{"type":"metricsets","beat":"metricbeat"},"system":{"process":{"username":"TREND\\ext_tienjingw","memory":{"size":10719232,"rss":{"bytes":12054528,"pct":0.0007},"share":0},"cmdline":"\\??\\C:\\windows\\system32\\conhost.exe 0x4","pid":5008,"state":"running","pgid":0,"name":"conhost.exe","ppid":12500,"cpu":{"total":{"pct":0.0},"start_time":"2017-07-28T06:49:10.435Z"}}},"metricset":{"module":"system","name":"process","rtt":120466},"beat":{"name":"tw-tienjingwang","hostname":"tw-tienjingwang","version":"5.5.1"},"@timestamp":"2017-07-28T06:51:06.919Z","type":"metricsets"}
{"@metadata":{"type":"metricsets","beat":"metricbeat"},"type":"metricsets","system":{"process":{"state":"running","pgid":0,"memory":{"rss":{"bytes":28434432,"pct":0.0017},"share":0,"size":18292736},"ppid":12500,"username":"TREND\\ext_tienjingw","cmdline":"metricbeat.exe","pid":14616,"cpu":{"total":{"pct":0.0623},"start_time":"2017-07-28T06:49:44.744Z"},"name":"metricbeat.exe"}},"metricset":{"module":"system","name":"process","rtt":120466},"beat":{"name":"tw-tienjingwang","hostname":"tw-tienjingwang","version":"5.5.1"},"@timestamp":"2017-07-28T06:51:06.919Z"}
Those events are emitted at same time of "2017-07-28T06:51:06.919Z". My analysis team have requested that I should group those messages by timestamp into one big message. Moreover, I have found the docs of Metricbeat have suggested such possibility:
Rather than containing a single metric, each event created by Metricbeat contains a list of metrics.
But I haven't been successful. I have searched around documents and full config files. But I haven't found any possiblity of bundle related metric into one message.
If possible, I would like this to be done in metricbeat config itself, as my team does not use logstash for aggregation nor elasticsearch.
Any pointer is appreciated, thank you.