Hi,
I have events indexed with logstash
I have a template defined in elasticsearch
Indexing only works sometimes after a new installation of my ELK.
Most of the times ES complains that mapping can't change from nested to non nested.
[2017-01-17T19:52:52,173][DEBUG][o.e.a.b.TransportShardBulkAction] [2s9hNaW] [events-2017.01.17][3] failed to execute bulk item (index) index {[events-2017.01.17][event/system/counter/pbuf][AVmt_T6yvOHW3FxTTZA5], source[{"sequence_number":73,"event_tags":[{"value":"nacl-st-pew5x9-ppu-server-b45nkwxfzwiy","key":"originhost"}],"@timestamp":"2017-01-17T19:52:52.014Z","data":{"counters":[{"name":"n_lost_packets","value":0},{"name":"n_sent_bytes","value":1701361752},{"name":"n_received_packets","value":5166146},{"name":"n_sent_packets","value":10418914},{"name":"n_ce_bytes","value":0},{"name":"n_received_bytes","value":284144254}],"resources":[{"identity":"bearer_path_pm","type":"name"},{"identity":"1","type":"ue_bearer_id"},{"identity":"","type":"pm_id"},{"identity":"1","type":"path_identifier"},{"identity":"2017-01-17 20:52:52.167567","type":"time"}]},"port":41432,"@version":"1","host":"10.68.32.189","producer":"1bda868e172de179c9d7e1b9955655fc059fe810","eventtype":"system/counter/pbuf","type":"event/system/counter/pbuf","timestamp":"2017-01-17T19:52:52.55313611Z"}]}
java.lang.IllegalArgumentException: object mapping [event_tags] can't be changed from nested to non-nested
at org.elasticsearch.index.mapper.ObjectMapper.doMerge(ObjectMapper.java:430) ~[elasticsearch-5.0.0.jar:5.0.0]
I can not see why my template is not working properly.
Can someone help me?
Before I start indexing I have an no event indexes:
curl -XGET 'http://elasticsearch:9200/_cat/indices?pretty'
green open .kibana Ic1CJibdQcKXm94W-UVc3w 1 0 62 362 296.3kb 296.3kb
My template looks like this:
curl -XGET 'http://elasticsearch:9200/_template/e*?pretty'
{
"eventbus" : {
"order" : 0,
"template" : "events-",
"settings" : {
"index" : {
"number_of_replicas" : "0"
}
},
"mappings" : {
"event" : {
"dynamic_templates" : [
{
"notanalyzed" : {
"mapping" : {
"index" : "not_analyzed",
"type" : "string"
},
"match_mapping_type" : "string",
"match" : ""
}
}
],
"properties" : {
"sequence_number" : {
"type" : "long"
},
"event_tags" : {
"type" : "nested"
},
"data" : {
"type" : "object"
},
"producer" : {
"index" : "not_analyzed",
"type" : "keyword"
},
"eventtype" : {
"type" : "keyword"
},
"timestamp" : {
"format" : "date_time",
"type" : "date"
}
}
}
},
"aliases" : { }
}
}
My data looks like this:
{
"sequence_number" => 73,
"event_tags" => [
[0] {
"value" => "nacl-st-pew5x9-ppu-server-b45nkwxfzwiy",
"key" => "originhost"
}
],
"@timestamp" => 2017-01-17T19:52:52.014Z,
"data" => {
"counters" => [
[0] {
"name" => "n_lost_packets",
"value" => 0
},
[1] {
"name" => "n_sent_bytes",
"value" => 1701361752
},
[2] {
"name" => "n_received_packets",
"value" => 5166146
},
[3] {
"name" => "n_sent_packets",
"value" => 10418914
},
[4] {
"name" => "n_ce_bytes",
"value" => 0
},
[5] {
"name" => "n_received_bytes",
"value" => 284144254
}
],
"resources" => [
[0] {
"identity" => "bearer_path_pm",
"type" => "name"
},
[1] {
"identity" => "1",
"type" => "ue_bearer_id"
},
[2] {
"identity" => "",
"type" => "pm_id"
},
[3] {
"identity" => "1",
"type" => "path_identifier"
},
[4] {
"identity" => "2017-01-17 20:52:52.167567",
"type" => "time"
}
]
},
"port" => 41432,
"@version" => "1",
"host" => "10.68.32.189",
"producer" => "1bda868e172de179c9d7e1b9955655fc059fe810",
"eventtype" => "system/counter/pbuf",
"type" => "event/system/counter/pbuf",
"timestamp" => "2017-01-17T19:52:52.55313611Z"
}
My list of indexes after a while. The event index is created but not populated.
curl -XGET 'http://elasticsearch:9200/_cat/indices?pretty'
green open events-2017.01.17 pIfhTieCRwqOS0P9Cvi2Kw 5 0 0 0 650b 650b
Isn't "event_data" nested in the example above?
If so, why there an attempt to change to non nested?
Thanks
Mathias