I rolled back to 5.2.0 just for logstash. It doesn't get stuck in the retry loop it just drops the message. This was the only thing I found that worked until they find a solution.
I did the same thing.. it is quieter but stops after 20-30 minutes. I now just have a script restarting logstash every 15 minutes until this is resolved.
I'm receiving around 700 documents/s. I don't know if volume has anything to do with it. I don't think it does.
Logstash 5.x will show request body is required due a bug that is fixed in Logstash 6.x
[2018-04-30T10:40:16,767][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>400, :url=>"http://localhost:9200/_bulk", :body=>"{\"error\":{\"root_cause\":[{\"type\":\"parse_exception\",\"reason\":\"request body is required\"}],\"type\":\"parse_exception\",\"reason\":\"request body is required\"},\"status\":400}"}
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.