# Input File Path ignores directories and gathering all the files

**URL:** <https://discuss.elastic.co/t/input-file-path-ignores-directories-and-gathering-all-the-files/168853>\
**Category:** Logstash\
**Created:** [February 18, 2019, 3:01pm UTC](https://discuss.elastic.co/t/input-file-path-ignores-directories-and-gathering-all-the-files/168853 "2019-02-18T15:01:44Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![torny](https://avatars.discourse-cdn.com/v4/letter/t/278dde/32.png) [@torny](https://discuss.elastic.co/u/torny)\
**Post date:** [February 18, 2019, 3:01pm UTC](https://discuss.elastic.co/t/input-file-path-ignores-directories-and-gathering-all-the-files/168853/1 "2019-02-18T15:01:44Z")

</div>

Hello,

I have a problem with Input -\> File plugin.  
On a fresh install I created index of bunch of csv files located in one directory. Works fine.  
Then I made neighboring directory with other csv files set. Made new config and new index. Unfortunately new index includes new files and old files from the first directory despite of explicit path in config.  
Where I'm wrong?

Configs:

cat /etc/logstash/conf.d/first.conf

```
   input {
    file {
    path => ["/home/admin-local/logs/*.csv"]
    start_position => "beginning"
    sincedb_path => "/dev/null"
      }
    }

     filter {
      csv {
       separator => ","
       columns => ["Date", "Build Number", "Subsystem", "Duration"]
         convert => {
         "Date" => "date_time"
         "Build Number" => "integer"
         "Duration" => "integer"
         }
     }
    }

    output {
     elasticsearch {
     hosts => "http://localhost:9200"
     index => "first-01"
    }
   }

```

cat /etc/logstash/conf.d/test.conf

```
input {
  file {
    path => ["/home/admin-local/test/*.csv"]
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
          csv {
           separator => ","
           columns => ["Date", "Build Number", "Subsystem", "Duration"]
             convert => {
             "Date" => "date_time"
             "Build Number" => "integer"
             "Duration" => "date_time"
             }
         }
}

output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "time-01"
  }
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 18, 2019, 4:04pm UTC](https://discuss.elastic.co/t/input-file-path-ignores-directories-and-gathering-all-the-files/168853/2 "2019-02-18T16:04:13Z")

</div>

If you have configured logstash with '-f /etc/logstash/conf.d' then it will concatenate all of the configuration files in that directory, read events from the inputs and send them to all of the outputs.

You could use a different pipeline for each configuration, or you could use conditionals based on the path.

---

<div class="post-metadata">

**Author:** ![torny](https://avatars.discourse-cdn.com/v4/letter/t/278dde/32.png) [@torny](https://discuss.elastic.co/u/torny)\
**Post date:** [February 18, 2019, 4:20pm UTC](https://discuss.elastic.co/t/input-file-path-ignores-directories-and-gathering-all-the-files/168853/3 "2019-02-18T16:20:56Z")

</div>

Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 18, 2019, 4:21pm UTC](https://discuss.elastic.co/t/input-file-path-ignores-directories-and-gathering-all-the-files/168853/4 "2019-03-18T16:21:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.