# Input file plugin doesn't process multiple input files on intensive logging in any of them
**URL:** https://discuss.elastic.co/t/input-file-plugin-doesnt-process-multiple-input-files-on-intensive-logging-in-any-of-them/103871
**Category:** Logstash
**Created:** [October 13, 2017, 10:07am UTC](https://discuss.elastic.co/t/input-file-plugin-doesnt-process-multiple-input-files-on-intensive-logging-in-any-of-them/103871 "2017-10-13T10:07:58Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@vkakhnych](https://discuss.elastic.co/u/vkakhnych)
#### Post date: [October 13, 2017, 10:07am UTC](https://discuss.elastic.co/t/input-file-plugin-doesnt-process-multiple-input-files-on-intensive-logging-in-any-of-them/103871/1 "2017-10-13T10:07:58Z")
Tested on logstash 2.3, centos6.
Simply logstash config:
* * *
```
input {
file {
path => ["/tmp/*-running.log"]
}
}
output {
stdout { codec => json_lines }
}
```
* * *
And simply logs popolate script:
* * *
```
#!/bin/sh
while true
do
DATE=$(/bin/date +'%Y-%m-%d@%T')
echo "$DATE batch1" >> /tmp/Test1-running.log
echo "$DATE batch2" >> /tmp/Test2-running.log
# sleep 1
done
```
* * *
I've expected to got in the output lines from both logs in serial order but get content only from one of them:
...
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.216Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.219Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.219Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.221Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.221Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.225Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.225Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.227Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.227Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.229Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.229Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.230Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
{"message":"2017-10-13@09:24:40 batch2","@version":"1","@timestamp":"2017-10-13T09:26:13.230Z","path":"/tmp/Test2-running.log","host":"localhost.localdomain"}
...
When `tail -f /tmp/*-running.log` output content from the both logs fine at the same time.
But if I remove a comment from `sleep 1` in the above script (logging stands not so intensively) I get expected results from file plugin. Unfortunately in real life we have much more logs with very high intensively of logging.
Is there any way to get expected results from file plugin?
We can't use filrebeat because of using AWS SQS for output.
Thanks a lot!
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 17, 2017, 6:43pm UTC](https://discuss.elastic.co/t/input-file-plugin-doesnt-process-multiple-input-files-on-intensive-logging-in-any-of-them/103871/2 "2017-10-17T18:43:44Z")
Yeah, I think each file input reads the first file until it hits EOF, then it continues with the next and the next and the next, finally circling back to the first one. Would it be possible for you to generate multiple file input blocks instead of using an all-encompassing wildcard? That way you'll get multiple threads processing the input files.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 14, 2017, 6:43pm UTC](https://discuss.elastic.co/t/input-file-plugin-doesnt-process-multiple-input-files-on-intensive-logging-in-any-of-them/103871/3 "2017-11-14T18:43:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.