Hello,
I'm trying to integrate a csv files but i have troubles with date and number format.
Files contain the information below
2016-06-01,xxxxx,AD,NA,4.31
....
I use the csv filter to collect datas then i try to add a new field and convert it to a number.
1/ The conversion "string to float" gave me always a number = 0.
2/ I don't find the right way to parse the date with the format "yyyy-mm-dd" in order to match @timestamp
########################################
if [type] == "rating_country" {
csv {
#Date,Package Name,Country,Daily Average Rating,Total Average Rating
columns => ["timevalue", "PackageName","Country","DailyAverageRating","TotalAverageRating"]
separator => ","
}
mutate {
add_field => {"TotalAverageRatingFloat" => ["%{TotalAverageRating}"]}
}
mutate {
convert => { "TotalAverageRatingFloat" => "float" }
}
date {
match => [ "timevalue", "yyyy-MM-dd HH:mm:ss" ]
}
}
########################################
The data into elasticsearch
{
"_index": "rating_country-2016.06.20",
"_type": "rating_country",
"_id": "AVVuGrov59dQgZAfvsI0",
"_version": 1,
"_score": 1,
"_source": {
"message": "2016-06-01,xxxxxx,AD,NA,4.31",
"@version": "1",
"@timestamp": "2016-06-20T13:58:12.717Z",
"path": "/Users/Mathieu/Documents/logs/market/stats/ratings/ratings_xxxxx_201606_country.csv",
"host": "BTWNMCPS061",
"type": "rating_country",
"timevalue": "2016-06-01",
"PackageName": "xxxxxx",
"Country": "AD",
"DailyAverageRating": "NA",
"TotalAverageRating": "4.31",
"TotalAverageRatingFloat": 0
}
}