Yet another multiple type issue in ES 6.X. I am stumped

[rouble]

Gurus,

I am recreating some work originally done in ES 5.X and I am hitting the multiple type issue. The original work can be found in github here. For some reason, and I cant explain why, when inserting entries they getting tagged with an type "doc", even though, I am not explicitly doing that anywhere.

The original project creates an index mapping for ES 5.X as shown here. I changed this template to work on ES 6.X as follows:

{
    "template": "cucm-cdr*",
    "settings": {
        "number_of_shards": 5,
        "number_of_replicas": 1
    },
    "mappings": {
        "doc": {
            "properties": {
                "type": { "type": "keyword" },
				"cdrRecordType" : { "type" : "integer" },
				"globalCallID_callManagerId" : { "type" : "integer" },
				"globalCallID_callId" : { "type" : "integer" },
                            ...<SNIP>...
            }
        }
    }
}

And then we load data using the logstash file input plugin.

Here is the logstash config:

input {
        file {
                path =>"/path/to/cdr*"
                type => "cucm-cdr"
                start_position => "beginning"
                add_field => { "cucm_clustername" => "CUCM_Cluster_1" }
        }
}

The full logstash config is here.

Now, when we load files via logstash I get the error:
[2018-09-21T00:54:10,919][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"cucm-cdr-21.09.2018", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x12b7acb3>], :response=>{"index"=>{"_index"=>"cucm-cdr-21.09.2018", "_type"=>"doc", "_id"=>"-1F6-mUB7q-Yw141ZWFT", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [cucm-cdr-21.09.2018] as the final mapping would have more than 1 type: [cucm-cdr, doc]"}}}}

Where is the type doc getting set and more importantly how do I get rid of it? Thanks in advance!

[jakelandis]

I think your Logstash elasticsearch output needs to declare the document_type, it will send doc by default.

output{
  elasticsearch {
...
        index => "%{type}-%{+dd.MM.YYYY}"
        document_type   => "cucm-cdr"
        manage_template => false # since not using logstash's default mappings
  }
}

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-document_type

[rouble]

Thanks @jakelandis

That helped get me closer. But still running into one issue. The first time this ran, it ran fine. But I had some other config issues. So, I flushed/deleted the indexes and restarted logstash. My logstash has sincedb_path => "/dev/null" so it tries to load the files again. Now, I get a strange error, notice the _type is correct this time, but same multiple key error.

[2018-09-21T12:29:14,656][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"cucm-cdr-21.09.2018", :_type=>"cucm-cdr", :_routing=>nil}, #<LogStash::Event:0x7d11aa4a>], :response=>{"index"=>{"_index"=>"cucm-cdr-21.09.2018", "_type"=>"cucm-cdr", "_id"=>"VFL2_GUB7q-Yw141vk09", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [cucm-cdr-21.09.2018] as the final mapping would have more than 1 type: [cucm-cdr, doc]"}}}}

[jakelandis]

It's sending the right type now (at least for the posted snippet). I wonder if something else is feeding that index with the doc type. I would suggest to delete that index and from the config you updated, and from that Logstash you changed, temporarily change the index name, then see if the old index name comes back up (which means something else is writing to it with a different type).

[rouble]

So, here is what I did, and it works. I set the document_type to doc - which should be the default (i thought). If @jakelandis or someone else at elastic can explain why this works, it would be much appreciated. In fact, I dont understand why multiple types is even an issue here since nothing in this configuration is trying to set multiple types.

Here is what I did (based on the inspiration from @jakelandis) :

output {
  if [type] == "cucm-cdr" {
    elasticsearch { hosts => ["127.0.0.1:9200"]
                    index => "%{type}-%{+dd.MM.YYYY}"
                    document_type   => "doc"
                    manage_template => false # since not using logstash's default mappings
    }
  } else if [type] == "cucm-cmr"{
    elasticsearch { hosts => ["127.0.0.1:9200"]
                    index => "%{type}-%{+dd.MM.YYYY}"
                    document_type   => "doc"
                    manage_template => false # since not using logstash's default mappings
    }
  } else {
    elasticsearch { hosts => ["127.0.0.1:9200"]
                    index => "%{type}-%{+dd.MM.YYYY}"
                  }
  }
} 

Here is the original config that did not work:

elasticsearch { hosts => ["127.0.0.1:9200"]
                index => "%{type}-%{+dd.MM.YYYY}"
              }