# Generate output index from input filename

**URL:** <https://discuss.elastic.co/t/generate-output-index-from-input-filename/138339>\
**Category:** Logstash\
**Created:** [July 3, 2018, 8:45am UTC](https://discuss.elastic.co/t/generate-output-index-from-input-filename/138339 "2018-07-03T08:45:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![n1ur0](https://avatars.discourse-cdn.com/v4/letter/n/7ab992/32.png) [@n1ur0](https://discuss.elastic.co/u/n1ur0)\
**Post date:** [July 3, 2018, 8:45am UTC](https://discuss.elastic.co/t/generate-output-index-from-input-filename/138339/1 "2018-07-03T08:45:52Z")

</div>

Hi,

I'm trying to parse and integrate multiple .csv files into elasticsearch through logstash csv plugin. However, as I have around 4000 csv files, is there a way to make the same input csv filename the same as the elasticsearch output index? To give some context, every csv file starts with "Amp\ __.csv", where "_" is a integer. Any way to make the index name as "Amp\__" with "_" as the corresponded digit(s)?

> input {  
> file {  
> path =\> "/home/emanuel/ondas/amp/Amp\_\*.csv"  
> start\_position =\> "beginning"  
> sincedb\_path =\> "/dev/null"  
> }  
> }  
> filter {  
> csv {  
> separator =\> ","  
> columns =\> ["id", "Amplitude", "time", "dist"]  
> remove\_field =\> ["True"]  
> }  
> mutate {convert =\> ["id", "integer"] }  
> mutate {convert =\> ["Amplitude", "float"] }  
> mutate {convert =\> ["time", "float"] }  
> mutate {convert =\> ["dist", "float"] }  
> }  
> output {  
> elasticsearch {  
> hosts =\> "192.168.20.32:9200"  
> document\_type =\> "data"  
> index =\> "WHAT TO PUT HERE?"  
> }  
> stdout { codec =\> json\_lines }  
> }

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 3, 2018, 8:50am UTC](https://discuss.elastic.co/t/generate-output-index-from-input-filename/138339/2 "2018-07-03T08:50:49Z")

</div>

The input filename should end up in the `path` field so you can use e.g. a grok filter to extract parts of the path into a field. That field can then be referenced in the elasticsearch output's `index` option. This has been discussed many times before so please consult the archives.

Having one index per input file is probably misguided. Indexes have a fixed overhead and the cost of having 4000 indexes in an ES cluster is not insignificant.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2018, 8:50am UTC](https://discuss.elastic.co/t/generate-output-index-from-input-filename/138339/3 "2018-07-31T08:50:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.