Filter date doesn't work

Elastic Stack Logstash
1

Hello everyone,

I have a column called "Date et heure" that is formatted as [dd/MM/YY HH:mm] (ie 04/12/2019 17:45),
I am using a date filter in order to treat this as a date when passing to ElasticSearch but it doesn't work, for example for "04/12/2019 17:45" it only loads "4",

Here is my config:

#######################################################

filter
{
csv
{
separator => ","

	columns => ["Date et heure","Utilisateur","Code","Libelle evenement","Code retour","Application","Code site","Objet Start","Usage cache","Valeur avant modif","Valeur apres modif"]
}

	
mutate{

	convert => { 
		
		"Date et heure" => "string"
		"Utilisateur" => "string" 
		"Code" => "integer" 
		"Libellé évènement" => "string" 
		"Code retour" => "integer" 
		"Application" => "string" 
		"Code site" => "integer" 
		"Objet Start" => "string" 
		"Usage cache" => "string" 						
		"Valeur avant modif" => "string" 
		"Valeur après modif" => "string"	
	
	}
	
	#Gestion des accents
	rename => { "Libelle evenement" => "Libellé évènement"  
				"Valeur apres modif" => "Valeur après modif" }
					

}

date {	
	
	match => [ "Date et heure", "dd/MM/YY HH:mm" ] 
	
}

}

############################################################

2

The example you showed has a 4-digit year while you specified a 2-digit year in your pattern. Does it make a difference if you correct this? Does the date filter the populate the @timestamp field correctly?

3

thank you for your reply i have corrected the year number of digits but still the same,
I didn't understood your second question, can you be more clear pls ?

4

The date filter by default populates the @timestamp field (in UTC timezone)) and does not modify the source field. Does it get populated correctly?

5

I was surprised to find that YY will match a 4-digit year!

input { generator { count => 1 lines => [ '' ] } }
filter {
    mutate { add_field => { "Date et heure" => "04/12/2019 17:45" } }
    date {  match => [ "Date et heure", "dd/MM/YY HH:mm" ] }
}
output { stdout { codec => rubydebug { metadata => false } } }

gets me

"Date et heure" => "04/12/2019 17:45",
   "@timestamp" => 2019-12-04T17:45:00.000Z,
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.