Logstash find a date/time in a log entry

Elastic Stack Logstash
1

I've just started using logstash and I'm working on trying to parse a log and push it into solr using the solr_http plugin.

I successfully used a grok filter to pull the severity out as a separate field, but I can't seem to get the date to parse.

My expectation would be the date would be added to a field called mydate similar to severity.

Sample file record:

localhost-startStop-1 2019-06-08 05:08:21,497 DEBUG lking.spi.MetamodelGraphWalker Visiting attribute path : tableRating

Here is my config:

#Try to push data into solr

input {
        file {
			 path => "//serverpath/hibernate.log"
                             start_position => "beginning"
         }
    }
filter {
	  grok {
		match => { "message" => "%{LOGLEVEL:severity}"}
	  }
	  date {
		match => [ "mydate", "yyyy-MM-dd HH:mm:ss,SSS" ]
		target => "mydate"
	  }
	  
	}		
output {
            solr_http {
                    solr_url => "http://localhost:8080/solr/logStuff"
                    }        
	}

The result of this is that data gets pushed into solr, there is a separate field for 'severity'. But nothing for mydate. I have the log level turned on to TRACE and nothing seems to come out on concerning the mydate field

2

Interesting. I got his to work, but not how I expected. I added a second "grok" filter for the date.

I am still curious why the date filter didn't work though. Any insight is welcome.

Working filter:
#Try to push data into solr

input {
    file {
			 path => "//serverpath/hibernate.log"
                             start_position => "beginning"
         }
    }
filter {
	  grok {
		match => { "message" => "%{LOGLEVEL:severity}"}
	  }
	  
	  grok {
		match => { "message" => "%{TIMESTAMP_ISO8601:logdate}"}
	  }
	  
	}		
output {
            solr_http {
                    solr_url => "http://localhost:8080/solr/logStuff"
                    }        
	}
3

Nothing in your first configuration creates a field called mydate, so the date filter is a no-op.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.