# LogStash multiple files in single location

**URL:** <https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490>\
**Category:** Logstash\
**Created:** [July 10, 2017, 2:30pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490 "2017-07-10T14:30:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ShashiK](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@ShashiK](https://discuss.elastic.co/u/ShashiK)\
**Post date:** [July 10, 2017, 2:30pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490/1 "2017-07-10T14:30:13Z")

</div>

Logstash works for me when i have a direct file reference -  
When I use the single file reference in the logstash config-  
path =\> "C:\LogRepository\transaction\Web\HttpUser\_1312.csv"  
It works for me.  
However when i put in a generic reference -  
path =\> "C:\LogRepository\transaction\Web\*.csv"  
It does not index any files.  
Problem for me is that this location contains individual user transaction data and I cannot say how many files or specifically the name of the files in this location. - this is a direct data dump.  
So I am wondering if logstash file filter can support a tree structure, i.e. directory containing directories containing file(s).

---

<div class="post-metadata">

**Author:** ![CDR](https://avatars.discourse-cdn.com/v4/letter/c/d9b06d/32.png) [@CDR](https://discuss.elastic.co/u/CDR)\
**Post date:** [July 10, 2017, 2:42pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490/2 "2017-07-10T14:42:40Z")

</div>

If I am understanding your problem correctly you have multiple files under the directory C:\LogRepository\transaction\Web. To read in the multiple files with the ending .csv change your path to this:

`path => "C:\LogRepository\transaction\Web\*.csv"`

This is saying that it will read in any files under the directory Web that end in .csv. What you originally had Logstash interpreted as "read anything under the directory transaction that follows Web\*.csv"

If you have multiple directories under Web you can choose this for path:

`path => "C:\LogRepository\transaction\Web\**\*.csv"`

This means that Logstash will look under any directory under Web for files ending in .csv

[https://www.elastic.co/guide/en/logstash/2.2/plugins-inputs-file.html#plugins-inputs-file-path](https://www.elastic.co/guide/en/logstash/2.2/plugins-inputs-file.html#plugins-inputs-file-path)

---

<div class="post-metadata">

**Author:** ![ShashiK](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@ShashiK](https://discuss.elastic.co/u/ShashiK)\
**Post date:** [July 12, 2017, 5:46pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490/3 "2017-07-12T17:46:22Z")

</div>

Thanks Colton.  
It was a typo in my post.  
below is the exact entry i have in my conf file.

input {  
file {  
path =\> "C:\Shashi\Tools\LogRepository\transaction\Web\http\*.csv"  
type =\> "transactions"  
start\_position =\> "beginning"  
}  
}

filter {  
csv {  
separator =\> ","  
columns =\> ["Username", "InterfaceType", "UserType", "MessageType",  
"MessageTemplate", "StartTimeinMillis", "EndTimeinMillis",  
"StartTime", "EndTime"]  
}  
}

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "test\_data"  
document\_type =\> "transactions"  
}  
stdout {}  
}

Still the same issue......

---

<div class="post-metadata">

**Author:** ![CDR](https://avatars.discourse-cdn.com/v4/letter/c/d9b06d/32.png) [@CDR](https://discuss.elastic.co/u/CDR)\
**Post date:** [July 12, 2017, 6:00pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490/4 "2017-07-12T18:00:08Z")

</div>

So it is not reading in any of the files that are located in that directory with .csv endings? Or when you say that it is not indexing do you mean that the files are being read in but not indexed in elasticsearch?

---

<div class="post-metadata">

**Author:** ![ShashiK](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@ShashiK](https://discuss.elastic.co/u/ShashiK)\
**Post date:** [July 12, 2017, 6:01pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490/5 "2017-07-12T18:01:57Z")

</div>

i dont think it is reading because i do not see any entries in elasticsearch.  
Is there any log file i can enable to figure out what is going wrong....  
again i do not think there is any issue with any of the input files.  
Because when i remove \*.csv and put Useri.csv as a specific file name then the records get indexed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 9, 2017, 6:01pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490/6 "2017-08-09T18:01:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.