How may I get only ERROR related logs from my stack trace?

Elastic Stack Logstash
1

Currently I am developing an application where I need to retrieve only "ERROR" related logs and put them to elasticsearch index.

Here is my sample log:

2018-07-05 19:37:15,888 [6] ERROR <Description> - Error while ProcessRequest, Level: ERROR

I am bit confused about this. Should I use particular grok pattern in filter or I can handle this in input itself?

This is my current logstash pipeline configuration:

           input{
                    file{
                    path => "D:\ELK_Info\APEXDataService.log"
                    start_position => "beginning"
                    codec => multiline {
                          pattern => "^%{TIMESTAMP_ISO8601} "
                          negate => true
                          what => previous
                        }
                    }
                    }

    output {
    if[LEVEL] == "ERROR"
      http {
        url => "http://localhost:9200/indexName/log"
        http_method => "post"
        format => "json"
      }
    }

I have added multi line codec in my input.
So my question is, how may I filter this to get all the data of my log related to ERROR to my output?

2

You need to use a filter (grok or dissect) to extract the loglevel into a field of its own. Once that's done a conditional like

if[LEVEL] == "ERROR"

will work.

So my question is, how may I filter this to get all the data of my log related to ERROR to my output?

Your multiline configuration looks fine. I find it very hard to believe that Logstash only includes the lines up to "End of stack trace ..." in an event it produces.

Why are you using an http output to post to Elasticsearch?

3

Thanks for your suggestion, will try to implement the same.

4

By the way, I want to create a document under a particular index and that's why I am sending the output to HTTP?
You can let me know how may I improve this.

5

By the way, I want to create a document under a particular index and that's why I am sending the output to HTTP?

The elasticsearch output plugin has configuration options for choosing the index.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.