Create grok for two different kind of logs

Elastic Stack Logstash
1

Hello Can someone Please guide me how to create grok for two different kind of logs.
A help would be highly appreciated.

2019/05/01 00:52:51.301 < Success
2019/05/01 00:53:04.443 > GET http://www.testenvironment.com/smsnew.asp?From_Number=123456&To_Number=256789f&Message=&Receive_Date=20190501&Receive_Time=1253&status=helloworld&message_type=psm.cli&source_location=PK&TSMC=

For the time I know but I am unable to separate furthur

Regards

3

Have you looked the example in the documentation showing how to specify multiple patterns to match different types of logs?

4

@Christian_Dahlqvist thanks for the response
Actually both types of logs are repeating in a single log file

5

That is what that example cover.

6

You can use the below format to grok pattern for two different kind of logs

input {
    file {
        path => ["/test/*"]  #file1.log & file2.log in folder
        start_position => "beginning"
    }
}

filter {
    grok {
        match => {"path" => "%{GREEDYDATA}/%{GREEDYDATA:type}"}
    }
    if [type] == "file1.log" {
         grok {
            match => { ... }
        }
    }
    else if [type] == "file2.log" {
         grok {
            match => { ... }
        }
    }
}

Please update me if you have any query or concern.

7

Hey @kirangavali
Both type of logs are in a single log file

8

You want to say there is only one single log file?

9

yes,

Both type of logs are in a single log file

10

Hey @shrikantgulia

In order to filter multiple patterns using grok, you simply need to use multiple "match" patterns as shown below:

input { stdin { } }

filter {
  grok {
       match => {"message" => "\A%{WORD:Success}"}
       match => {"message" => "\A%{WORD:GET}"}
  }

}

output {
  stdout { codec => rubydebug }
}

Also, in order to filter dates from messages in a single log file, you would need to use match pattern in multiple filters as shown below:

input { stdin { } } 

filter { 
  grok { 
    match => { "message" => "%{COMBINEDAPACHELOG}" } 
  } 
  date { 
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:Ss Z" ] 
  } 
} 

output { 
  elasticsearch { hosts => ["localhost:9200"] } 
  stdout { codec => rubydebug } 
}

Please let me know if you have further query or concern.

2 Likes
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.