Unable to convert a column with Date and time format from string to date

Elastic Stack Logstash
1

Hi... my csv file contains closedDate with only date(e.g: 9/3/2019) and createdDate with date and time(e.g: 9/4/2019 06:40).
I'm able to convert "closedDate" field to date format but "createdDate" field is remaining as string.

Below is the filter I used.

filter {
csv {
separator => ","
autogenerate_column_names => true

skip_header => true

    skip_empty_columns => true
 

  columns => ["IncidentNumber","CreatedDate","Status","ClosedDate","Prods","State"]

}
mutate {convert => ["Prods", "integer"]}
date { match => ["ClosedDate", "MM/dd/yyyy"]
target => "ClosedDate" }
date { match => ["CreatedDate", "MM/dd/yyyy HH:mm:ss"]
target => "CreatedDate" }

}

Kindly help me
Thank you

2

Your example date does not include seconds, so remove the ":ss". The pattern in a date filter has to exactly match the entire field.

3

thank you Badger

It is working. But i have got another issue difference in time
e.g: closed_date : 09-05-2018
output shows "sep 4 2018 @ 11:30:00.000"

created_date : 09-05-2018 14:26
o/p "sep 05 2018 @ 02:26:00.000"

Where is the issue ?

4

You may need to use the timezone option on the date filter. elasticsearch always stores timestamps as UTC. You use the timezone option to tell logstash what the local timezone on the log file is. kibana will translate the UTC timestamps from elasticsearch into the browsers local timezone.

5

Thank you Badger

Will try with the date filter.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.