Correct date format for data ingestion

Elastic Stack Logstash
1

Hi guys,

Im trying to ingest some data and I encountered a problem with the correct date assignment.

The following sample of data looks like this:

"price_eur" => 4922.28,
"engine_power" => 122,
"maker" => "mazda",
"seat_count" => 5,
"mileage" => 59000,
"fuel_type" => "gasoline",
"stk_year" => "None",
"date_created" => "2015-11-14 19:00:48.141345+00",
"door_count" => 5,
"message" => "mazda;6;59000;2007;2261;122;;;None;man;5;5;gasoline;2015-11-14 19:00:48.141345+00;2016-01-27 20:40:15.46361+00;4922.28\r",
"path" => "/data/obs-dbb2/ElasticSearch_Data/cars.csv",
"body_type" => nil,
"host" => "sysm-aiops-elk-master1",
"color_slug" => nil,
"engine_displacement" => "2261",
"model" => "6",
"transmission" => "man",
"@version" => "1",
"manufacture_year" => "2007",
"date_last_seen" => "2016-01-27 20:40:15.46361+00",
"@timestamp" => 2020-01-31T08:46:42.733Z

The problem is time_created and date_last_seen - it is still ingesting it as String, but I would like to ingest it as long (just like the field @timestamp, which is ingested correctly).

In my configuration file, I tried various types, the latest time format i tried, looks like this:

   date {
    match => ["date_created", "yyyy-MM-dd HH:mm:ss.SSSZZ"]
   target => "@date_created"

Is there a way to make it indexing time_created, and date_last_seen just like timestamp ?

Thank you

Martin

2

Hey
You need to convert it to a date in logstash to ensure its in the right format, then map that field in elasticsearch in your index to a date format using an index template or the mapping api.
Best to ask how to do that in the elasticsearch forum if you're unsure. Alternatively, the documentation is pretty comprehensive :slight_smile:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.