# Multiple CSV files need to upload for same column names

**URL:** <https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012>\
**Category:** Logstash\
**Created:** [January 4, 2018, 5:37am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012 "2018-01-04T05:37:23Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hari\_Baskar](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\
**Post date:** [January 4, 2018, 5:37am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/1 "2018-01-04T05:37:24Z")

</div>

Hi,

I need to upload monthly wise csv data(which contains the same columns) using logstash. So how to upload that and create an index.

Thanks in Advance.

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [January 4, 2018, 5:48am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/2 "2018-01-04T05:48:07Z")

</div>

Hi Hari ,

You can use csv filter plugin for the same.  
Read the multiple csv file (in your case monthly basis) from the same location using file input plugin and use csv filter to process the data as per columns and send it to output to elasticsearch (if are storing it in es)

```
input {
  file {
    path => "/path_to_your_csv_files/*.csv"
    start_position => "beginning"
  }
}
filter {
  csv {
      separator => ","
#Your column names here
     columns => ["column1","column2".......]
  }
}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "your_index_name_here"
  }

}
```

---

<div class="post-metadata">

**Author:** ![Hari\_Baskar](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\
**Post date:** [January 4, 2018, 7:01am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/3 "2018-01-04T07:01:46Z")

</div>

Hi Rohan,

Thanks for replied. I have already tried this option, But it won't work. So, Please tell some other option. Actually, I'm using elastic in Windows. So, Is any problem?

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [January 4, 2018, 7:14am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/4 "2018-01-04T07:14:54Z")

</div>

> [@Hari\_Baskar](#):
>
> Thanks for replied. I have already tried this option, But it won't work. So, Please tell some other option. Actually, I'm using elastic in Windows. So, Is any problem?

It should work , what is the issue that your facing here? and there is no problem at all if you are using elasticsearch on windows.

---

<div class="post-metadata">

**Author:** ![Hari\_Baskar](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\
**Post date:** [January 4, 2018, 8:02am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/5 "2018-01-04T08:02:00Z")

</div>

Hi Rohan,

I have uploaded images which i am getting issues. First image is config file and second screen is error when running the by that format.

 ![cog](https://us1.discourse-cdn.com/elastic/original/3X/8/3/83e6fbc517d227ad5556bcc66edaae09c42477a6.png) ![drop](https://us1.discourse-cdn.com/elastic/original/3X/c/8/c82bd8bfc6ee4d32e9ee366742bf4bae4a278ab4.png)

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [January 4, 2018, 9:14am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/6 "2018-01-04T09:14:35Z")

</div>

Is the data getting indexed into elasticsearch?

as output screenshot only shows document\_type deprecation warnings  
Kindly validate your config file

---

<div class="post-metadata">

**Author:** ![Hari\_Baskar](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\
**Post date:** [January 4, 2018, 9:24am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/7 "2018-01-04T09:24:45Z")

</div>

No, It's not indexed. Because I have used path like="\path\*.csv" . So, It's not getting indexed. I have checked the config already. If I mention single csv file name in path, It works, But when I using \*.csv, Its showing error. So, Help me to resolve.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 6, 2018, 10:23am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/8 "2018-01-06T10:23:09Z")

</div>

> If I mention single csv file name in path, It works, But when I using \*.csv, Its showing error. So, Help me to resolve.

Have you tried using forward slashes instead of backslashes in the filename pattern? Also, make sure you include the drive letter.

---

<div class="post-metadata">

**Author:** ![Hari\_Baskar](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\
**Post date:** [January 8, 2018, 1:05pm UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/9 "2018-01-08T13:05:49Z")

</div>

Hi,  
Thanks for reply. I have tried that also, It won't work. I'm using windows. Here how to define path?

---

<div class="post-metadata">

**Author:** ![Hari\_Baskar](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\
**Post date:** [January 9, 2018, 11:17am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/10 "2018-01-09T11:17:00Z")

</div>

Finally, I got the solution. I have used "\foldername1\foldername2/\*". It works fine.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 6, 2018, 11:17am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012/11 "2018-02-06T11:17:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
