Date Formatting of

Elastic Stack Logstash
1

I need parse date in logstash of format [03/Apr/2016:10:35:57 +0530] .
This is from Weblogic Access logs.

How to do it.

Regards,
Kunal

2

The first configuration example at https://www.elastic.co/guide/en/logstash/current/config-examples.html is very very similar to what you want to do.

3

Still I a facing issue.Please look at the following.
---Configuration File----
input{
file
{
path=>"/home/kunal/ELK/logstash-2.2.2/bin/date.txt"
start_position => beginning
ignore_older => 0
}
}
filter
{
grok{
#patterns_dir=> " /home/kunal/ELK/logstash-2.2.2/patterns/patterns"
match => {"Date"=>"[%{HTTPDATE:time}]"}
}

date{

match => ["time","dd/mm/yyyy:HH:mm:ss Z"]

}

}
output{
stdout{
codec=> rubydebug
}
}

e.g date format file contents

[15/May/2016:12:16:23]
[15/May/2016:12:16:23]

=======================================================

Logstash Output

/ELK/logstash-2.2.2/bin$ ./logstash agent -f httpDate.conf
Settings: Default pipeline workers: 1
Logstash startup completed
{
"message" => "[15/May/2016:12:16:23]",
"@version" => "1",
"@timestamp" => "2016-06-17T07:07:38.102Z",
"path" => "/home/kunal/ELK/logstash-2.2.2/bin/date.txt",
"host" => "localhost",
"tags" => [
[0] "_grokparsefailure"
]
}

================================================

I think I am doing some small mistake..I am begginer to ELK.

Any suggestion on the same.

4

Three problems:

  • The field with the date is in your case message, not time as you've configured your date filter.
  • The message field contains square brackets but your date pattern doesn't.
  • The date pattern includes "Z" but there's no timezone in the message field.
1 Like
5

Hi Magnus Bäck,

Thank you very much.....

Last 3 suggestion solved it.

After commenting GROK pattern every things worked fine.

Now If I am having log pattern which is combination of Text / data as well as date information ,then i need to use grok as well ...m i right.?..e.g weblogic or websphere or apache logs.....

Regards,
Kunal

6

Yes, grok is a common tool to extract fields from text. Most Logstash configuration will contain at least one grok filter.