# How to get logstash to read/process multiple log files at the same time?

**URL:** https://discuss.elastic.co/t/how-to-get-logstash-to-read-process-multiple-log-files-at-the-same-time/301710
**Category:** Logstash
**Created:** [April 6, 2022, 7:05am UTC](https://discuss.elastic.co/t/how-to-get-logstash-to-read-process-multiple-log-files-at-the-same-time/301710 "2022-04-06T07:05:15Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Shreesh\_Narayanan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreesh_narayanan/32/87312_2.png) [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)
#### Post date: [April 6, 2022, 7:05am UTC](https://discuss.elastic.co/t/how-to-get-logstash-to-read-process-multiple-log-files-at-the-same-time/301710/1 "2022-04-06T07:05:15Z")

</div>

It seems that logstash doesnt read multiple files in the directory at the same time , what am i missing ? how do i get logstash to read all three files here ?

```auto
root@dev-elk-app02:/app/input# lsof *
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 7218 root 118r REG 253,0 38579356903 131074 22.223.11.125_ddc.log
java 7218 root 119r REG 253,0 38579356903 131074 22.223.11.125_ddc.log
java 7218 root 120r REG 253,0 38579356903 131074 22.223.11.125_ddc.log
root@dev-elk-app02:/app/input# ls
189.21.70.65_ddc.log 22.223.11.125_ddc.log 20.221.11.46_ddc.log

```

Here's a sample input

```auto
root@dev-elk-app02:/app/input# cat /etc/logstash/conf.d/tests/logstash-dissect_r_line_mle.conf 
input {
file {
path => "/app/input/*_ddc.log"
start_position => "beginning"
mode => "read"
close_older => 0
sincedb_path => "/dev/null"
check_archive_validity => "true"
stat_interval => "1ms"
discover_interval => 1
file_chunk_size => 3276800
max_open_files => 100000
file_completed_action => "log_and_delete"
file_completed_log_path => "/app/processed/output.txt"
}
#stdin {}
      }
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 6, 2022, 4:49pm UTC](https://discuss.elastic.co/t/how-to-get-logstash-to-read-process-multiple-log-files-at-the-same-time/301710/2 "2022-04-06T16:49:33Z")

</div>

If you enable log.level trace you should see which files it is checking/opening/closing.

---

<div class="post-metadata">

### Author: ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)
#### Post date: [April 6, 2022, 7:34pm UTC](https://discuss.elastic.co/t/how-to-get-logstash-to-read-process-multiple-log-files-at-the-same-time/301710/3 "2022-04-06T19:34:33Z")

</div>

How are you starting Logstash?

This is most likely due to the user permissions on each file. Currently it is `root` but if you are starting Logstash as a service it should be `logstash`. Or the group.

If you are starting Logstash via command line you might just need to add `sudo`.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 4, 2022, 7:34pm UTC](https://discuss.elastic.co/t/how-to-get-logstash-to-read-process-multiple-log-files-at-the-same-time/301710/4 "2022-05-04T19:34:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.