We have had several cases recently of logstash not being able to restart all of its configured pipelines. We are using the aggregation filter and it should write out any aggregation maps that are in-progress when logstash is stopped. It does this, but sometimes 1 or 2 of them appear to be incomplete. When logstash is started up, those pipelines are aborted with this error:
Pipeline aborted due to error {:pipeline_id=>"pipeline2", :exception=>#<EOFError: End of file reached>
The only solution seems to be deleting the bad files, and losing all their stored events.
Anyone know why this happens or have any ideas on what to try?
We are on logstash 7.6.2.