# Logstash Not processing file in a folder simaltaneously

**URL:** https://discuss.elastic.co/t/logstash-not-processing-file-in-a-folder-simaltaneously/155898
**Category:** Logstash
**Created:** [November 8, 2018, 1:55pm UTC](https://discuss.elastic.co/t/logstash-not-processing-file-in-a-folder-simaltaneously/155898 "2018-11-08T13:55:49Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![hary1987](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hary1987/32/37425_2.png) [@hary1987](https://discuss.elastic.co/u/hary1987)
#### Post date: [November 8, 2018, 1:55pm UTC](https://discuss.elastic.co/t/logstash-not-processing-file-in-a-folder-simaltaneously/155898/1 "2018-11-08T13:55:49Z")

</div>

Hello All,

Logstash is not processing all files in a folder simultaneously/parallelly. It is processing one after the other.

I am using one configuration file.

Example:

input {  
file {  
path =\> "/inputfiles/record\*.txt"  
start\_position =\> "beginning"  
close\_older =\> 1  
}  
}

inputfiles folder has three files  
record1,record2,record3.

I have used sysout to see how it is indexing and picking file. it is processing one file after other file.

For example 3 million records it took 30 minutes. Even when i placed three files with three million records in a folder it took around 90 minutes. Of course that is because it processes one after the other.

Thanks

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [November 16, 2018, 4:23pm UTC](https://discuss.elastic.co/t/logstash-not-processing-file-in-a-folder-simaltaneously/155898/2 "2018-11-16T16:23:44Z")

</div>

The docs say this:

> In some cases it is useful to be able to control which files are read first, sorting, and whether files are read completely or banded/striped. Complete reading is **all of** file A then file B then file C and so on. Banded or striped reading is **some of** file A then file B then file C and so on looping around to file A again until all files are read. Banded reading is specified by changing [`file_chunk_count`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-file_chunk_count) and perhaps [`file_chunk_size`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-file_chunk_size). Banding and sorting may be useful if you want some events from all files to appear in Kibana as early as possible.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 14, 2018, 4:36pm UTC](https://discuss.elastic.co/t/logstash-not-processing-file-in-a-folder-simaltaneously/155898/3 "2018-12-14T16:36:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
