Define filter - to use second field as timestamp and add year


(Sharon Sasporta) #1

My logs entries look like that:

<<DEBUG>>  **[Mar 01 14:55:18]** [[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'] [CM] [OMS] (BasicParameter.getEffectiveDate) Exit
<<DEBUG>>  [Mar 01 14:55:18] [[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'] [CM] [OMS] (BasicParameter.getExpirationDate) Enter

I need to use the date as timestamp.

Following the code from my logstash.conf file:

            multiline {
                       patterns_dir => "/users/mpswrk1/LogStash/impls/patterns/patterns"
                       pattern => "^\<\<%{LOGLEVEL}\>\> "
                       negate => true
                       what => "previous"
            }
            grok {
                  match => { "message" => "\<\<%{LOGLEVEL:severity}\>\>  \[%{PARTTIMESTAMP:timestamp}\] \[\[%{DATA:status}\] %{DATA:execute_thread}\] \[%{WORD:source_app}\] \(%{JAVACLASS:method}\) %{DATA:exception_method_description}\: \(%{DATA:error_code}\) %{DATA:error_description}%{JAVASTACKTRACEPART:java_class_stack}" }
                  patterns_dir => "/users/mpswrk1/LogStash/impls/patterns/patterns"
            }
            date {
                  match => [ "timestamp" , "MMM dd, yyyy hh:mm:ss aa" ]
            }

*parttimestamp is a private
PARTTIMESTAMP %{MONTH} %{MONTHDAY} %{TIME

Regards
Sharon.


(Magnus Bäck) #2

What's the question, exactly? Logstash will guess which year a log is from so you might not have to do anything.


(Sharon Sasporta) #3

It doesn't work. I am getting in the timestamp the current date, of inserting into the elasticsearch.

Maybe the date { match {......} } line is wrong?


(Magnus Bäck) #4

Yes, you have to remove "yyyy" since there's no year in your timestamp. If the date filter fails it'll log an error message that usually points to where the problem is.


(Sharon Sasporta) #5

Will check and update. Thanks!


(Sharon Sasporta) #6

still not working, and I don't understand the reason.

This is how I defined it in the conf file:

            grok {
                  match => { "message" => "\<\<%{LOGLEVEL:severity}\>\>  \[%{PARTTIMESTAMP:timestamp}\] \[\[%{DATA}\] %{DATA}\] \[%{DATA:application}\] \[%{DATA}\] \(%{JAVACLASS}\) %{CISCO_REASON}.%{JAVACLASS}: \(%{DATA:error_code}\)" }
                  patterns_dir => "/etc/logstash/patterns"
            }
            date {
                  match => ["timestamp" , "MMM dd HH:mm:ss"]
                  #add_field => { "Status" => "Matched"}
                  #remove_field => ["timestamp"]
            }

(Sharon Sasporta) #7

Working


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.