I had to migrate my Elasticsearch 7.0.1 in a docker container from ssd to hdd.
After that I can see some filebeat log inputs are very late.
I use filebeat to send Nginx logs using a log file input for multiple files and use my custom pipeline for parsing them. Data from access log files with little write load appears in Elasticsearch very fast. But data from busy log files (from the same filebeat and the same server) is late for 4-6 hours.
How does the concurency work in this case? I don't know which parameters I need to tune. Is it possible to solve this problem by simply increasing count of workers?