Convert one filed from CSV column header to time stamp when loading csv to elasticsearch


#1

no. count sub-category ranking date
1 23 sd N 2016-10-23
The above is a simple snippet for CSV data. I use logstash to ship the data to elasticsearch vis csv filter.
I want to make date as the timestamp on elasticsearch, is it possible to do that? Thank you!


(Hans) #2

HI @shilong, Yes it is possible, however your information only contains a date without any time?

Lets say your CSV has the following fields, e.g.:

1 23 sd N 2016-10-23
......
Nr = 1
Events = 23
SD = sd
Direction = N
Date = 2016-10-23

Then here is an example of how you can convert this to a date the field Date = 2016-10-23:

csv {
columns => [
"Nr",
"Events",
"SD",
"Direction",
"Date"
]
separator => " "
remove_field => ["message"]
}
date {
locale => "en"
match => ["Date", "YYYY-MM-dd", "ISO8601"]
timezone => "Africa/Windhoek"
target => "@timestamp"
add_field => { "debug" => "timestampMatched"}
}

I hope this helps


#3

Thanks @Hans
It doesn't work out really.
I am wondering if the data like @timestamp tends to be November 27th 2016, 20:54:52.157, not just date value like 2016-11-27.


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.