Logstash errors on filebeat received apache logs

Elastic Stack Logstash
1

After an upgrade from 6.2 to 6.3, I am now getting indexing errors for apache logs.

[2018-06-13T16:12:18,782][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"apache-2018.06.13", :_type=>"apache", :_routing=>nil}, #LogStash::Event:0x4db324f8], :response=>{"index"=>{"_index"=>"apache-2018.06.13", "_type"=>"apache", "_id"=>"tidr-2MB8IC994mx1_US", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [host]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:970"}}}}}
[2018-06-13T16:12:18,790][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"apache_error-2018.06.13", :_type=>"apache_error", :_routing=>nil}, #LogStash::Event:0x74e641ce], :response=>{"index"=>{"_index"=>"apache_error-2018.06.13", "_type"=>"apache_error", "_id"=>"tydr-2MB8IC994mx1_US", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [host]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:175"}}}}}

Any ideas?

2

What's the mapping of the host field? Use the ES get mapping API.

3 
{
  "apache-2018.06.13": {
    "mappings": {
      "apache": {
        "host": {
          "full_name": "host",
          "mapping": {
            "host": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        }
      }
    }
  }
}
4

It may be related to this issuse.

Metricbeat host namespace causes mapping conflict when used with Logstash

I installed the version of filebeat back to 6.2.4. it works fine.

5

A few years ago there was also such discussion about LS, Beat compatibility.

Copy beat.hostname into host

6

I reverted to filebeat 6.2.4 and now the insertions are taking place without issue / as expected. I'm trying to understand the host field issue as described in the link you posted.

7

Duplicate: Logstash errors after upgrading to filebeat-6.3.0

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.