Pipeline workers

The Elasticsearch is full of data but I don't think its all the data.

When I graph the data via Kibana on # of records per file per logstash @timestamp, I get huge reads per file at one at a time and then it moves to the next file and does not make it back to the first file for over 20 mins or longer. I would expect Logstash to be able to tail all files all at the same time if it has enough memory and pipeline workers.

So I have kept trying to scale up Logstash to handle all files all at the same time but so far I have not been able to see that happening.