Logstash is not processing all files in a folder simultaneously/parallelly. It is processing one after the other.
I am using one configuration file.
path => "/inputfiles/record*.txt"
start_position => "beginning"
close_older => 1
inputfiles folder has three files
I have used sysout to see how it is indexing and picking file. it is processing one file after other file.
For example 3 million records it took 30 minutes. Even when i placed three files with three million records in a folder it took around 90 minutes. Of course that is because it processes one after the other.