# Logstash join two csv files, one to many data

**URL:** https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019
**Category:** Logstash
**Created:** [July 31, 2019, 2:52am UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019 "2019-07-31T02:52:12Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![genice\_ladroma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/genice_ladroma/32/51291_2.png) [@genice\_ladroma](https://discuss.elastic.co/u/genice_ladroma)
#### Post date: [July 31, 2019, 2:52am UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019/1 "2019-07-31T02:52:12Z")

</div>

**Hi,**

**I was able to follow instructions on joining 2 csv files from this link**

> [@Logstash - how to configure CSV filter for joining 2 CSV files based on a common field and send the joined log to ES](https://discuss.elastic.co/t/logstash-how-to-configure-csv-filter-for-joining-2-csv-files-based-on-a-common-field-and-send-the-joined-log-to-es/167125/4):
>
> translate { destination =\> "[@metadata][lookup]" dictionary\_path =\> "/home/user/foo.csv" field =\> "COMMIT\_SHA1" } dissect { mapping =\> { "[@metadata][lookup]" =\> "%{FILE\_CHANGED};%{FILE\_TYPE};%{FILE\_METHOD}" } }

**However, this is just for joining one to one data, e.g.**  
_file1.csv_  
_id,name_  
_1,john_  
_2,jonathan_

_file2.csv_  
_id,lastname_  
_1,wick_  
_2,smith_

**this results to**  
_1,john,wick_  
_2,jonathan,smith_

**I would like to know if there's a way we join 2 csv file with one to many data, e.g.**  
_file1.csv_  
_id,name_  
_1,john_  
_2,jonathan_

_file2.csv_  
_id,lastname_  
_1,wick_  
_1,summers_  
_2,smith_  
_2,winters_

**which will result to**  
_1,john,wick_  
_1,john,summers_  
_2, jonathan, smith_  
_2, jonathan, winters_

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 31, 2019, 1:19pm UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019/2 "2019-07-31T13:19:23Z")

</div>

Use file1.csv as the dictionary in a translate filter. logstash does not care that there are many events that lookup the same entry.

---

<div class="post-metadata">

### Author: ![James\_Daily](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_daily/32/51985_2.png) [@James\_Daily](https://discuss.elastic.co/u/James_Daily)
#### Post date: [August 9, 2019, 1:48am UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019/3 "2019-08-09T01:48:15Z")

</div>

Can an id ever have two entries in file1, eg:

1 John  
1 Jerry  
2 Alice

?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 9, 2019, 12:23pm UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019/4 "2019-08-09T12:23:22Z")

</div>

Under the hood translate is using a hash, so when it loads the file it will not object, but only Jerry will be included in the translation of 1.

---

<div class="post-metadata">

### Author: ![James\_Daily](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_daily/32/51985_2.png) [@James\_Daily](https://discuss.elastic.co/u/James_Daily)
#### Post date: [August 9, 2019, 2:17pm UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019/5 "2019-08-09T14:17:20Z")

</div>

I meant to direct my question to @genice_ladroma 🙂

If file1 doesn't have unique id's, then the dictionary approach will not work.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 9, 2019, 2:27pm UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019/6 "2019-08-09T14:27:29Z")

</div>

Agreed, many to many will not work, but one to many should.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 6, 2019, 2:27pm UTC](https://discuss.elastic.co/t/logstash-join-two-csv-files-one-to-many-data/193019/7 "2019-09-06T14:27:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
