Breaking Changes With Logstash 6.0

Elastic Stack Logstash
1

I just managed to get my Logstash config working with v6.0... but I don't understand why it's working and when it doesn't.

Original config (does not work):

elasticsearch {
  hosts => ["rockserver1.lan"]
  index => "bro-%{[@meta][event_type]}-%{+YYYY.MM.dd}"
  document_type => "doc"
  manage_template => false
}

Gives:
[2018-01-21T18:23:18,031][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"bro-network", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x1cb38cc0>], :response=>{"index"=>{"_index"=>"bro-network", "_type"=>"doc", "_id"=>"eE71GWEBaJkcJV312X_a", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [bro-network] as the final mapping would have more than 1 type: [doc, network]"}}}}

Working Config:

elasticsearch {
  hosts => ["rockserver1.lan"]
  index => "bro"
  document_type => "doc"
  manage_template => false
}

Why does removing the dynamic indexing fix the problem and how does that relate to the error message?

2

A buddy helped me out. What was happening was that on my build, there was an elasticsearch template for the data. It was setting the type to network so when I tried to come in and set it to doc, doc was the second type.

Snippit:

{
  "index_patterns": "bro-network*",
  "mappings": {
    "network": {
      "properties": {
        "@meta": {
          "properties": {
            "geoip_orig": {
              "properties": {
                "ip": {
                  "type": "ip"
                },
                "location": {
                  "type": "geo_point"
                }

Changing network to doc fixed the problem.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.