# How to process csv data from multiple folders using file beat and create separate index and move it to elastic search

**URL:** https://discuss.elastic.co/t/how-to-process-csv-data-from-multiple-folders-using-file-beat-and-create-separate-index-and-move-it-to-elastic-search/292641
**Category:** Elasticsearch
**Created:** [December 22, 2021, 6:07am UTC](https://discuss.elastic.co/t/how-to-process-csv-data-from-multiple-folders-using-file-beat-and-create-separate-index-and-move-it-to-elastic-search/292641 "2021-12-22T06:07:43Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![data123](https://avatars.discourse-cdn.com/v4/letter/d/77aa72/32.png) [@data123](https://discuss.elastic.co/u/data123)
#### Post date: [December 22, 2021, 6:07am UTC](https://discuss.elastic.co/t/how-to-process-csv-data-from-multiple-folders-using-file-beat-and-create-separate-index-and-move-it-to-elastic-search/292641/1 "2021-12-22T06:07:43Z")

</div>

Hi Team,

Our requirement is like this...

we have log files stored in different folders under master folder as csv format.  
we want to send this to Elasticsearch using file beats as a part of automation.

we cant find solution how to handle multiple csv files using file beat from multiple folders and pull data from those folders and create different index for each csv file and move to Elasticsearch and visualize in kibana. its like streaming the log files.

Plz let us know we want to use logstash for this scenario

can anybody please help how to approach this process  
Thanks in advance

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [December 22, 2021, 6:52am UTC](https://discuss.elastic.co/t/how-to-process-csv-data-from-multiple-folders-using-file-beat-and-create-separate-index-and-move-it-to-elastic-search/292641/2 "2021-12-22T06:52:49Z")

</div>

I did something like this in the past.

Have a look at:

> **[Enriching Your Postal Addresses With the Elastic Stack - Part 1](https://www.elastic.co/blog/enriching-your-postal-addresses-with-the-elastic-stack-part-1)**

This is using Logstash.

I also have another "version" where I'm using filebeat for this and ingest pipelines.

The code is available at:

> **[GitHub - dadoonet/bano-elastic: Demo repo on how to use Bano with elastic to...](https://github.com/dadoonet/bano-elastic)**
>
> Demo repo on how to use Bano with elastic to do data enrichment - GitHub - dadoonet/bano-elastic: Demo repo on how to use Bano with elastic to do data enrichment

There's a video here which might help.

> **[Enriching postal addresses with Elastic stack by David Pilato](https://speaker.pilato.fr/UAqIx5)**
>
> Come and learn how you can enrich your existing data with normalized postal addresses with geo location points thanks to open data and BANO project.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 19, 2022, 6:53am UTC](https://discuss.elastic.co/t/how-to-process-csv-data-from-multiple-folders-using-file-beat-and-create-separate-index-and-move-it-to-elastic-search/292641/3 "2022-01-19T06:53:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
