Timestamp format in logstash 1.5.6

Elastic Stack Logstash
1

My log have this format 11-30-2016 12:13:57 INFO - Mapped "{[/transaction/get-account-history],methods=[POST],params=[],headers=[],consumes=[],produces=[],custom=[]}"

then how do i include timestamp format in filter ?

my second query is
i included path => ["/Users/tcstsb3/Downloads/log/*.log"] contain many logs

How to combine all logs and sort based on time
In kibana i want to show like
Example :
11-30-2016 12:13:57 INFO hello ....(from log1)
11-30-2016 12:13:57 INFO hai ..... (from log2)
11-30-2016 12:13:58 INFO msg .....(from log1)

2

then how do i include timestamp format in filter ?

Use a grok filter to extract the timestamp, loglevel, and possibly other pieces of the string into separate fields. Then use a date filter to parse the timestamp string into the @timestamp field.

How to combine all logs and sort based on time

As long as your date filter works that's exactly how Kibana will behave.

3

Thank you so much buddy :slight_smile:

i tried
filter {

grok {
  match => { "message" => "%{COMBINEDAPACHELOG}" }
 }

date {
   match => [ "timestamp" , "MM-dd-yyyy HH:mm:ss Z" ]
 }

}

doesnt work correctly..
can you correct my format

4

Your log isn't an Apache log so COMBINEDAPACHELOG obviously won't work. Perhaps the grok constructor web site can help you create a usable grok expression.

5

mine apache log only buddy

just for sample i mentioned like
11-30-2016 12:13:57 INFO hello ....(from log1)
11-30-2016 12:13:57 INFO hai ..... (from log2)
11-30-2016 12:13:58 INFO msg .....(from log1)

but im using apache log only

how can i change my timestamp format so that it will sort based on timestamp

6

The log examples you have shown aren't Apache combined logs so COMBINEDAPACHELOG won't work, period.

7

this my apache web server log only buddy

11-30-2016 12:13:57 INFO - Mapped "{[/transaction/get-account-history],methods=[POST],params=[],headers=[],consumes=[],produces=[],custom=[]}"

8

As I've already said:

  • That's not an Apache combined log.
  • The grok constructor web site can help you create a usable grok expression.

Over and out.

9

Thank you buddy, i removed that
my msg like 11-30-2016 12:13:57 INFO...
im trying to modify the timestamp of log

filter {
grok {
match => [ "message" , "MM-dd-YYYY HH:mm:ss" ]
}
date {
match => [ "timestamp" , "MM-dd-YYYY HH:mm:ss" ]
target => "@timestamp"
}
}

but json still showing like this

"message" => "11-10-2016 12:13:57 INFO - FrameworkServlet 'dispatcher': initialization completed in 39 ms",
"@version" => "1",
"@timestamp" => "2017-03-24T05:59:14.365Z",
"host" => "Vishnu-Prasad.local",
"path" => "/Users/tcstsb3/Downloads/log/starbuck-batch.log",
"tags" => [
[0] "_grokparsefailure"
]

timestamp format not changed ..

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.