# Logstash configuration file in joining two csv files based on primary key

**URL:** https://discuss.elastic.co/t/logstash-configuration-file-in-joining-two-csv-files-based-on-primary-key/292792
**Category:** Logstash
**Created:** [December 23, 2021, 9:47am UTC](https://discuss.elastic.co/t/logstash-configuration-file-in-joining-two-csv-files-based-on-primary-key/292792 "2021-12-23T09:47:35Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Dhamu-143](https://avatars.discourse-cdn.com/v4/letter/d/f6c823/32.png) [@Dhamu-143](https://discuss.elastic.co/u/Dhamu-143)
#### Post date: [December 23, 2021, 9:47am UTC](https://discuss.elastic.co/t/logstash-configuration-file-in-joining-two-csv-files-based-on-primary-key/292792/1 "2021-12-23T09:47:35Z")

</div>

Hi,  
I'm not able to join the two csv files based on the primary key(same value or same field).  
I'm taking two csv input files that are shown below  
Student Table(Table 1)  
Stdid,sname,fee  
121,john,10000  
123,glenn,12000  
124,James,14000  
125,nick,15000  
126,jimmy,16000

Library table(Table 2)  
stdid,sanme,bookname  
121,john,english  
122,max,java  
123,glenn,c++  
124,smith,c  
127,Root,python

The above are two files with common field (stdid) we need to do join transformation based on that and merge the files based on the conditions .  
For that we need a config file to load the data into elastic through logstash.

please try to help with this.  
Regards,  
A Reddy.

---

<div class="post-metadata">

### Author: ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)
#### Post date: [December 23, 2021, 2:36pm UTC](https://discuss.elastic.co/t/logstash-configuration-file-in-joining-two-csv-files-based-on-primary-key/292792/2 "2021-12-23T14:36:03Z")

</div>

This type of use case is usually better for a traditional relational database. If you setup a small MySQL database and then use those CSV files as [external tables](https://dev.mysql.com/doc/refman/8.0/en/innodb-create-table-external.html) then you can query the MySQL database using the logstash JDBC input  
`select t1.Stdid, t1.sname, t1.fee, t2.sanme, t2.bookname from Student t1, Library t2 where t1.Stdid = t2.stdid`  
and then ingest the data into Elasticsearch.

Alternatively you could load all of your data from the Student table into Elasticsearch, then when you ingest the Library table you can query the Student Elasticsearch index and enhance your Library data into another index, but that's not as elegant. Like I said this kind of work is best done by a relational database.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 20, 2022, 2:36pm UTC](https://discuss.elastic.co/t/logstash-configuration-file-in-joining-two-csv-files-based-on-primary-key/292792/3 "2022-01-20T14:36:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
