How can i get aggregated count of particular errors

I want to get number of times particular error occured on my server.After parsing logs look as below mentioned.

"path" => "/home/utkarsh/sample.log2016_07_21.txt",
"time" => "00:00:05,626",
"host" => "Laptop-012",
"@version" => "1",
"message" => "00:00:05,626 ERROR [HealthPackageCronService] - Unable to mark package as expired: ",
"level" => "ERROR",
"@timestamp" => 2018-04-18T08:18:30.602Z,
"gdata" => "[HealthPackageCronService] - Unable to mark package as expired: "

I want to get how many times a particular error have occured,so that i'm able to find which errors are occuring frequently.Please help

And the gdata field contains the error that you want to track? Make sure the field is a keyword field by adjusting the index template you're using, then use Elasticsearch to aggregate on that field and count the frequency of various messages.

thanks @magnusbaeck
yes. gdata field i want to track and get a count of particular error from that.
As i am new to logstash.Can you please tell me how i can achieve this.

Okay, but what part of my previous answer was unclear?

my requirement is to get aggregation of gdata on a exportable file for mailing purpose (i can done manually) on daily basis
so i don't want elastic search in this scenario

Since Logstash doesn't store any events on its own you'll have to build something outside Logstash. You could e.g. have Logstash dump the parsed log events to a file and then at whatever interval you like run a script that parses that file and tabulates the number of errors and produces an output file.

There is one plugin in logstash aggregate filter plugin.Can you please tell me can i use that plugin to achieve my objective or what is its usage?

You can probably use the aggregate filter, but if you do it'll continuously give you aggregated counts. If you want to produce reports once a day this isn't very practical since you'd have to aggregate the aggregates.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.