# Does Logstash process single file or multiple files simultaneously?
**URL:** https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108
**Category:** Logstash
**Created:** [October 27, 2016, 7:57am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108 "2016-10-27T07:57:03Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)
#### Post date: [October 27, 2016, 7:57am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/1 "2016-10-27T07:57:03Z")
How does the logstash process multiple log files ??? What I mean is, does the logstash processes one file at a time (and then move on to another file) or multiple files simultaneously ???
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 27, 2016, 8:07am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/2 "2016-10-27T08:07:38Z")
Logstash was designed to monitor and process files as they are actively being written to, so it will process multiple files simultaneously.
---
### Author:  [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)
#### Post date: [October 27, 2016, 8:13am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/3 "2016-10-27T08:13:45Z")
So, why does the .sincedb file is updated only when a file is completely processed ?
I am using logstash to process the logs (not to monitor it).
I had put 2 log files of size 100 Mb each in the folder being monitored by logstash. Information about first file was entered in .sincedb file only when logstash finished processing it. Same thing happened with second file.
---
### Author:  [@Kamaal\_Hassan](https://discuss.elastic.co/u/Kamaal_Hassan)
#### Post date: [October 27, 2016, 8:23am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/4 "2016-10-27T08:23:44Z")
if you really want to process logs from multiple files on the Go, meaning if file 2 updates often then file1, Better use filebeat. it monitors multiple files, and send responses to logstash .
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 27, 2016, 8:28am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/5 "2016-10-27T08:28:50Z")
> So, why does the .sincedb file is updated only when a file is completely processed ?
I don't know, but it's a different question from "will Logstash process files in parallel".
> I had put 2 log files of size 100 Mb each in the folder being monitored by logstash. Information about first file was entered in .sincedb file only when logstash finished processing it. Same thing happened with second file.
I think Logstash processes files in a given file input serially but if you have multiple file inputs the files will be processed in parallel.
---
### Author:  [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)
#### Post date: [October 27, 2016, 8:45am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/6 "2016-10-27T08:45:37Z")
> [@Kamaal\_Hassan](#):
>
> Better use filebeat. it monitors multiple files, and send responses to logstash
Will I be able to get the Logstash reading progress of each individual file while the file is being processed ? (I have not used Filebeat before)
---
### Author:  [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)
#### Post date: [October 27, 2016, 8:46am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/7 "2016-10-27T08:46:36Z")
> [@magnusbaeck](#):
>
> it's a different question from "will Logstash process files in parallel
I will take care of this next time. 🙂
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 27, 2016, 8:47am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/8 "2016-10-27T08:47:23Z")
> Will I be able to get the Logstash reading progress of each individual file while the file is being processed ?
Filebeat keeps a registry file that's similar in spirit to Logstash's but in a different format.
---
### Author:  [@Kamaal\_Hassan](https://discuss.elastic.co/u/Kamaal_Hassan)
#### Post date: [October 27, 2016, 8:55am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/9 "2016-10-27T08:55:44Z")
Yeah Mayank. File beat will monitor all logs included in the yml. But for first run it also spans the logs sequentially. After that , any update in listed files will be pushed to logstash. in logstash u need to add a filter for each file incase the GROKs are different for each file.
---
### Author:  [@Kamaal\_Hassan](https://discuss.elastic.co/u/Kamaal_Hassan)
#### Post date: [October 27, 2016, 8:57am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/10 "2016-10-27T08:57:15Z")
Yeah I saw the registry , its similar to logstash. but for first run , I guess its always sequential spanning files is a apt solution. beat monitors all files and push any new lines from last read.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:32am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108/11 "2017-07-06T04:32:24Z")