Could Not Index Event to Elasticsearch Error?

Elastic Stack Elasticsearch
1

Hi I am sending information from Logstash to Elasticsearch. The Elasticsearch is working fine but when I try to send it information to it from a log file it created the index for it but does not receive any of the information not sure why, help would be greatly appreciated.

this is the error I receive

[2017-11-06T14:09:16,033][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"testing1234", :_type=>"OldLog", :_routing=>nil}, 2017-11-06T20:09:15.729Z xlrd23 10.04.14 20:07:02.53 (5211) FM 5037 [17 Oct 03], Bfed:23, what: 0, fdfdf: 0], :response=>{"create"=>{"_index"=>"testing1234", "_type"=>"OldLog", "_id"=>"AV-S840O3TjZGp2ACWuq", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Mixing up field types: class org.elasticsearch.index.mapper.core.LongFieldMapper$LongFieldType != class org.elasticsearch.index.mapper.ip.IpFieldMapper$IpFieldType on field host"}}}}}

2

Looks like you have a conflict, where the host field is a long but you want to change it to an IP somewhere.

What is the current mapping of that field in the index?

3

Hey Warkolm thank you for replying.
Im not sure what you mean by what is the mapping in the index sorry I am new to the ELK stalk.
but for the host field it appears as the following in stdout.

"host" => "xlrd23 "

that is our server host name of where I have logstash installed

4

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats and APM feel left out! :wink:

Run this - curl -XGET ESHOSTIP:9200/testing1234/_mapping and then paste the output. Please use the </> button to format it in the post as well :slight_smile:

5

Ok for mappings this is what I got,

 "testing1234" : {
"mappings" : {
  "OldLog" : { },
  "event" : {
    "properties" : {
      "category" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "component" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "compressed" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "event_category" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "event_source" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "host" : {
        "type" : "ip"
      },
      "isSynced" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "isValidFormat" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "logType" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "number_of_hops" : {
        "type" : "short"
      },
      "payload_size" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "port" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "priority" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "route_string_1" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "route_string_2" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "route_string_3" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "sourceType" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "source_dest" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog5424_app" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog5424_host" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog5424_msgid" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog5424_pri" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog5424_proc" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog5424_ts" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog5424_ver" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog_facility" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog_facility_code" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog_severity" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "syslog_severity_code" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "traceDestinationHost" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "traceDestinationPort" : {
        "type" : "short"
      },
      "traceSourceHost" : {
        "type" : "string",
        "index" : "not_analyzed"
      },
      "traceroute" : {
        "properties" : {
          "dest_ip" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "dest_name" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "probes" : {
            "properties" : {
              "ipaddr" : {
                "type" : "ip"
              },
              "name" : {
                "type" : "string",
                "index" : "not_analyzed"
              },
              "rtt" : {
                "type" : "double"
              }
            }
          }
        }
      },
      "tzKnown" : {
        "type" : "string",
        "index" : "not_analyzed"
      }
    }
  }
}

}
}

6

Ok, so you have this already defined in the mapping;

But the output is a string;

Hence the error.

To fix this you need to delete the index, update the mapping so that field is a keyword and then run Logstash again.

1 Like
7

I have been beating myself trying to solve this for almost a week now thank you so much Warkolm! sucks but I cannot update the mappings the elasticsearch instance is administered by another group within our organization but all I did was remove the host field in my Logstash config file. Using the following

filter {
mutate { remove_field => [ "host"] }
}

but without your help I would have never figured out that that was the problem. Thank you for all your help!

8

No worries.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.