Transform datetime to @timestamp

Elastic Stack Logstash
1

I'm trying to use a datetime field (startTime) from the input xml logs and use it as @timestamp but something is not properly set as it idenfies the field as string but is not able to apply the transformation leaving the current time as @timestamp.

This is a sample of the date 2019-09-24T13:15:16.134+02:00

I tested with the folowing formats but none of them seems to work:

  • "yyyy-MM-dd'T'HH:mm:ss'.'SSSZZ",
  • "yyyy-MM-dd'T'HH:mm:ss'.'SSSZ",
  • "yyyy-MM-dd'T'HH:mm:ss','SSSZZ",
  • "yyyy-MM-dd'T'HH:mm:ss','SSSZ",
  • "yyyy-MM-dd'T'HH:mm:ss.SSSZZ",
  • "yyyy-MM-dd'T'HH:mm:ssSSSZ",
  • "yyyy-MM-dd'T'HH:mm:ssSSSZZ",
  • "yyyy-MM-dd'T'HH:mm:ssSSSZ"

What should be the correct format for transforming the datetime ?

date {
	match => [ "startTime", "yyyy-MM-dd'T'HH:mm:ss'.'SSSZZ" ]
}

Thanks in advance

2 
date { match => [ "ts", "YYYY-MM-dd'T'HH:mm:ss.SSSZZ" ] }

will convert "2019-09-24T13:15:16.134+02:00" to 2019-09-24T11:15:16.134Z

3

Thanks for your quick response @Badger.

Changin the "ts" for the name of the field that has the date I have a tag _dateparsefailure.

   filter{
       xml {
           source => "message"
           store_xml => false
           remove_namespaces => true
           xpath =>[
               "//AdapterFrameworkData","recordsList"
           ]
       }       
       split {
           field => "recordsList"
       }
       xml {
           source => "recordsList"
           store_xml => false
           remove_namespaces => true
           xpath =>[
               "//messageKey/text()","messageKey",
               "//interface/namespace/text()","namespace",
               "//interface/name/text()","name",
               "//status/text()","status",
               "//startTime/text()","startTime"
           ]
       }
	   date { match => [ "startTime", "YYYY-MM-dd'T'HH:mm:ss.SSSZZ" ] }	   
	   mutate
	   {
		  remove_field => [ "message" ]
	   }
   }
4

Is it an array? Does [startTime][0] work?

5

It is. I was looking into the wrong direction.

Thanks a lot.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.