Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"52801f35-d64d-4585-b698-5628baa8552d", :_index=>"logstash-summary-2019.03.06", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x2f805b51>], :response=>{"index"=>{"_index"=>"logstash-summary-2019.03.06", "_type"=>"doc", "_id"=>"52801f35-d64d-4585-b698-5628baa8552d", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Mixing up field types: class org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType != class org.elasticsearch.index.mapper.KeywordFieldMapper$KeywordFieldType on field orderID"}}}}}
I am trying to fix this issue. I recently learned about the mapping files and changes between ES2 to ES5 and ES6. My original issue was that I had a field that was of type string, which caused incorrect query results in elasticsearch so in return, I had incorrect data. However, I've been trying to replace this field to be a type;keyword OR type:string with index:not_analyzed, but whenever I apply that change on the template. I keep getting that error. So the template applies for all indices with the matching template name correct? I am not sure why when I delete all my indices and delete my template on ES and redeploy a fresh version. I still get the error above. Is there some hidden thing I am forgetting to change? Could it be logstash causing this field to be Text?
{
"template": "logstash-summary-*",
"settings": {
"index.refresh_interval": "5s"
},
"mappings": {
"logs": {
"_all": {
"enabled": true,
"omit_norms": true
},
"dynamic_templates": [
{
"message_field": {
"match": "message",
"match_mapping_type": "string",
"mapping": {
"type": "string",
"index": "analyzed",
"omit_norms": true
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"type": "string",
"index": "not_analyzed",
"ignore_above": 256
}
}
}
],
"properties": {
"orderID": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
This is my new template. Originally under orderID, I just have type:string, no index parameter. I get the same error when I change it to type:keyword (I understand ES5 and ES6 use type:keyword as a type:string with index:not_analyzed)
This is the template that I pull from the index
{
"logstash-summary-2019.03.06": {
"mappings": {
"logs": {
"_all": {
"enabled": true,
"norms": false
},
"dynamic_templates": [
{
"message_field": {
"match": "message",
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"index": "analyzed",
"norms": false,
"omit_norms": true,
"type": "string"
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"ignore_above": 256,
"index": "not_analyzed",
"norms": false,
"type": "string"
}
}
}
],
"properties": {
"@timestamp": {
"type": "date",
"include_in_all": false
},
"@version": {
"type": "keyword",
"include_in_all": false
},
"geoip": {
"dynamic": "true",
"properties": {
"ip": {
"type": "ip"
},
"latitude": {
"type": "half_float"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "half_float"
}
}
},
"orderID": {
"type": "keyword"
}
}
}
}
}
}