How can i get aggregated count of particular errors

Elastic Stack Logstash
1

hi,
I want to get number of times particular error occured on my server.After parsing logs look as below mentioned.

<
{
"path" => "/home/utkarsh/sample.log2016_07_21.txt",
"time" => "00:00:05,626",
"host" => "Laptop-012",
"@version" => "1",
"message" => "00:00:05,626 ERROR [HealthPackageCronService] - Unable to mark package as expired: ",
"level" => "ERROR",
"@timestamp" => 2018-04-18T08:18:30.602Z,
"gdata" => "[HealthPackageCronService] - Unable to mark package as expired: "
}

I want to get how many times a particular error have occured,so that i'm able to find which errors are occuring frequently.Please help

2

And the gdata field contains the error that you want to track? Make sure the field is a keyword field by adjusting the index template you're using, then use Elasticsearch to aggregate on that field and count the frequency of various messages.

3

thanks @magnusbaeck
yes. gdata field i want to track and get a count of particular error from that.
As i am new to logstash.Can you please tell me how i can achieve this.

4

Okay, but what part of my previous answer was unclear?

5

my requirement is to get aggregation of gdata on a exportable file for mailing purpose (i can done manually) on daily basis
so i don't want elastic search in this scenario

6

Since Logstash doesn't store any events on its own you'll have to build something outside Logstash. You could e.g. have Logstash dump the parsed log events to a file and then at whatever interval you like run a script that parses that file and tabulates the number of errors and produces an output file.

7

There is one plugin in logstash aggregate filter plugin.Can you please tell me can i use that plugin to achieve my objective or what is its usage?

8

You can probably use the aggregate filter, but if you do it'll continuously give you aggregated counts. If you want to produce reports once a day this isn't very practical since you'd have to aggregate the aggregates.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.