Could not retrieve remote IP address for beats input

Elastic Stack Logstash
1

"Could not retrieve remote IP address for beats input."

I see this warning regularly. (even though the comment says it should never happen)

My setup is 6.0.0-rc1
beats are running on a remote server in docker containers.

ELK stack is deployed with this docker-compose repo:
decomposed/elk
It uses the official elastic docker images.

Any advice how to troubleshoot this error message?

2

More details:
In bursts of less than 0.1 seconds, I get around 30 of these errors:

[2017-10-23T14:59:17,540][WARN ][logstash.inputs.beats ] Could not retrieve remote IP address for beats input.
... 30 more...
[2017-10-23T14:59:17,549][WARN ][logstash.inputs.beats ] Could not retrieve remote IP address for beats input.

There are 3 events that were stashed at 14:59:17.019 - one ajax request (got a 200) and two gitlab runner requests (got 204s) All three have remote_ip fields with valid ips (one is a 172 address from another docker container on the same box)

Again I ask for help to troubleshoot/investigate, as the error does not explicitly tie itself to a specific request...

3

After updating to version 6 of the ELK stack I've had similar issues, along with documents not being able to be indexed because they have 2 document types (this seems to be related to old plugins not having been properly updated).

Have you found any fix for this?

4

Both errors happen for me too

[2017-11-16T13:38:41,327][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-2017.11.15", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x3f81832e>], :response=>{"index"=>{"_index"=>"filebeat-2017.11.15", "_type"=>"doc", "_id"=>"uH7WxF8BDa1ley8Gmc9y", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [filebeat-2017.11.15] as the final mapping would have more than 1 type: [log, doc]"}}}}
[2017-11-16T13:38:41,328][WARN ][logstash.inputs.beats    ] Could not retrieve remote IP address for beats input.
5

Same issue opened a ticket here: https://github.com/logstash-plugins/logstash-input-beats/issues/269

6

Going to: vim /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.2-java/lib/logstash/inputs/beats/message_listener.rb

And deleting:

begin
    hash.get("@metadata").put("ip_address", ctx.channel().remoteAddress().getAddress().getHostAddress())
  rescue #should never happen, but don't allow an error here to stop beats input
    input.logger.warn("Could not retrieve remote IP address for beats input.")
end

Fixes the issue.