Hi, logstash suddenly shows a quite strange behaviour, i think it's trying to parse a text field into a date field. But i really don't know how to interpret this message in
logstash-plain.log:
[2017-07-06T16:23:43,116][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"cons33-edicola", :_type=>"cons33", :_routing=>nil}, 2017-07-06T14:23:42.862Z HUelastic %{message}], :response=>{"index"=>{"_index"=>"cons33-edicola", "_type"=>"cons33", "_id"=>"AV0YSP12as7FI9I-fVyb", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Data]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"Corriere della Sera + Edizioni L...\""}}}}}
I just put my template as i always did, letting elasticsearch manage the date field, and it worked until today, but now this.
Here's my mapping:
curl -XPUT 192.168.136.10:9200/_template/cons33-edicola -d '
{
"template": "cons33-edicola",
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0,
"refresh_interval": "30s"
},
"mappings": {
"cons33": {
"properties": {
"IDMEDIA":{"type":"keyword"},
"ENTE": {
"type": "keyword"
},
"IDUSER": {
"type": "keyword"
},
"Anno": {
"type": "keyword"
},
"titolo": {
"type": "keyword"
},
"Data": {
"format": "yyyy-MM-dd",
"type": "date"
}
}
}
}
}
'
And this is my logstash conf:
input {
file {
path => "/home/../samlpe.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
columns => ["IDMEDIA", "ENTE", "IIDUSER", "titolo", "Anno", "Data"]
separator => ","
}
mutate {
convert => {
"IDMEDIA" => "string"
"ENTE" => "string"
"IDUSER" => "string"
"Anno" => "string"
"titolo" => "string"
}
}
mutate {
remove_field => ["message"]
}
}
output {
elasticsearch {
action => "index"
hosts => ["localhost"]
index => "cons33-edicola"
document_type => "cons33"
template_name => "cons33-edicola"
template_overwrite => false
manage_template => false
}
stdout {}
}
Has it ever occurred to you? Any explanation?
thank you