Logstash metrics

Good day everyone.

I have a problem configuring logstash to store metrics. So the goal is to have statistics based on
connected servers. I want to see how many messages each of my servers emit in a point of time.

looks like metrics plugin is a way to go, but i can't store this data in elasticseach in the way i want.

So far i came with this configuration:

filter {
  if [message] and [host] {
    metrics {
      meter => [ "%{host}" ]
      add_tag => "metric"
output {
     if "metric" in [tags] {
         file {
           path => "/tmp/test.out" 
	   codec => rubydebug {
		metadata => true


This is output

"@version" => "1",
"@timestamp" => "2017-03-17T11:22:05.227Z",
"message" => "el1",
"nlbint2" => {
"count" => 172,
"rate_1m" => 0.028768455154214556,
"rate_5m" => 0.08007273499832629,
"rate_15m" => 0.08472905443413455
"rfapp-test" => {
"count" => 67543,
"rate_1m" => 38.32375267123969,
"rate_5m" => 31.368829680900127,
"rate_15m" => 27.19415559842569

so when i put this to elasticsearch there is one message "el1" with the bunch of fields.

How i can make it look like @timestamp host count rate1m rate5m rate15m ?

Are you trying to measure Logstash throughput to troubleshoot? If not, I would use the count aggregation, split by Date Histogram then by server field. With that you can view messages per server at any interval you need.

The idea is to track log generation spikes from each server. So if we see that particular server starts to emit many messages there is a posibility that it feels bad.
Unfortunately not all logs delivered to elastic. Some of them stored in raw format on filesytem. But all of them still collected by logstash.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.