How to use grok pattern file in logstash filter

[abhishek007 (abhishek)]

Hi,
I have log with this pattern.
07/06/2018 09:42:39.607,T,someString,pool-9-thread-4,D:007,1528335759607,120

I want to write grok pattern using grok debugger, but could not succeed .
I found one grok pattern file on github. that has long list of patterns.

How can I use that to grok the above log pattern?
My test configuration is as below.

input { stdin { } }
output { stdout { codec => rubydebug } }
filter {

date {
match => ["loggingTime", "dd/MM/YYYY HH:mm:ss:SSSS"]
}
}

That gives below output.
{
"@timestamp" => 2018-06-08T09:44:50.615Z,
"@version" => "1",
"host" => "hostname",
"message" => "07/06/2018 09:42:39.607,T,someString,pool-9-thread-4,D:007,1528335759607,120"
}

What is meaning of the date filter when it does not create field "loggingTime" in the output?

May be i have mixed 2 different questions, but still... my requiement is I want to index timestamp under field loggingTime and I want to know how to use grok pattern file efficiently.

br,
Abhishek.

[Christian_Dahlqvist]

Have a look at this blog post which walks you through how to use Logstash and parse data in a couple of different ways.

[abhishek007 (abhishek)]

Hi,
My log line is 07/06/2018 08:42:39.607 INFO [pool-9-thread-3][c.s.t.d.r.DatabaseDictionaryProvider] log description message

Here is my LS config:

input { stdin { } }
output { stdout { codec => rubydebug } }
filter {

dissect {
mapping => {
"message" => "%{timestamp->} %{logLevel} [%{threadId}][%{logger}] %{logMessage}"
}

}

}

why is that timestamp not coming under same field. Why the time part is clubbing with the loglevel? whats wrong with this can anybody figure out?

{
"logger" => "c.s.t.d.r.DatabaseDictionaryProvider",
"@timestamp" => 2018-06-14T06:24:04.318Z,
"host" => "myHost.ind.com",
"timestamp" => "07/06/2018",
"logLevel" => "08:42:39.607 INFO",
"logMessage" => "log description message",
"@version" => "1",
"threadId" => "pool-9-thread-3",
"message" => "07/06/2018 08:42:39.607 INFO [pool-9-thread-3][c.s.t.d.r.DatabaseDictionaryProvider] log description message"
}

[Christian_Dahlqvist]

As you have a space between date and time you need to match both parts separately using append field notation. I suspect your pattern should start like this: %{timestamp} %{+timestamp} %{logLevel}