Csv filter match date field

Elastic Stack Logstash
1

I am trying to parse e csv. All works fine except for date field.

This is the format: 2020-02-21 18:25:37.254774
and this is my date code:

date {
    match => ["date", "yyyy-MM-dd HH:mm:ss.SSS"]
    target => "@timestamp"
}

This is full csv line:
csvlog "2020-02-21 18:25:37.254774" srv1 10.10.100.1 48619 192.168.25.50 57027 TCP

This is full logstash conf:

input {
            file {
             path => "/var/log/csv.log"
             start_position => "beginning"
           }
         }  
           filter {
                csv {
                    columns => [
                      "event.module",	
                      "date",		
                      "agent.hostname",	
                      "source.ip",		
                      "source.port",	
                      "destination.ip",	
                      "destination.port",	
                      "network.transport"
                    ]
            	add_tag => [ "%{event.module}" ]
            	add_field => {
            		"destination.address" => "%{destination.ip}"
            		"source.address" => "%{source.ip}"
            	}
                    separator => " "
                    remove_field => ["message"]
                }
                date {
                    match => ["date", "yyyy-MM-dd HH:mm:ss.SSS"]
                    target => "@timestamp"
                }
     }
2

remove target => "@timestamp" and it should work.

I did quick test and it did work

# cat stdin_test.conf
input { stdin { } }

filter {
   date { match => ["message", "dd/MMM/yyyy:HH:mm:ss"] }
}
output { stdout { codec => rubydebug } }

echo '21/Feb/2020:11:15:02' | /usr/share/logstash/bin/logstash -f stdin_test.conf

and I got

{
         "@version" => "1",
       "message" => "21/Feb/2020:11:15:02", ---> original time
    "@timestamp" => 2020-02-21T17:15:02.000Z  --> time in UTC
}
3

Thank You for the answer.
I found the problem but note the solution.

My date formate use a fraction of a second to long:
2020-02-21 18:25:37.254774 NOT WORK
2020-02-21 18:25:37.254 WORK

How can i truncate the date so the fraction of a second is smaller?

4

Why not use a matching pattern?...

match => ["date", "yyyy-MM-dd HH:mm:ss.SSSSSS"]
5

It works with yyyy-MM-dd HH:mm:ss.SSSSSS.

Thank you

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.