# Logstash - how to configure CSV filter for joining 2 CSV files based on a common field (mapping - one to many data) and ingest the data into single index

**URL:** https://discuss.elastic.co/t/logstash-how-to-configure-csv-filter-for-joining-2-csv-files-based-on-a-common-field-mapping-one-to-many-data-and-ingest-the-data-into-single-index/259994
**Category:** Logstash
**Created:** [January 1, 2021, 12:49pm UTC](https://discuss.elastic.co/t/logstash-how-to-configure-csv-filter-for-joining-2-csv-files-based-on-a-common-field-mapping-one-to-many-data-and-ingest-the-data-into-single-index/259994 "2021-01-01T12:49:48Z")
**Posts on this page:** 1
**Showing post:** 2

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 1, 2021, 4:27pm UTC](https://discuss.elastic.co/t/logstash-how-to-configure-csv-filter-for-joining-2-csv-files-based-on-a-common-field-mapping-one-to-many-data-and-ingest-the-data-into-single-index/259994/2 "2021-01-01T16:27:24Z")

</div>

You will need to reformat the csv1 file, but you can do it using a translate filter.

Use a file input to read csv2 and then use the following filters

```
    csv { autodetect_column_names => true }
    translate {
        dictionary_path => "/home/user/t.test/csv1.csv"
        field => "faculty_id"
        destination => "faculty_name"
    }

```

Then you would use a second translate filter that has a file that looks like

```
faculty_id,reporting_manager
1,R1
2,R2

```

Alternately, completely changing the format of csv1

```
faculty_id,JSON
1,"{""faculty_name"": ""AAA"", ""reporting_manager"": ""R1""}"
2,"{""faculty_name"": ""BBB"", ""reporting_manager"": ""R2""}"

```

then use these filters

```
    csv { autodetect_column_names => true }
    translate {
        dictionary_path => "/home/user/csv1.csv"
        field => "faculty_id"
        destination => "[@metadata][json]"
    }
    json { source => "[@metadata][json]" }
```

---

_[View the full topic](https://discuss.elastic.co/t/logstash-how-to-configure-csv-filter-for-joining-2-csv-files-based-on-a-common-field-mapping-one-to-many-data-and-ingest-the-data-into-single-index/259994)._