# Merging CSV files based on a common key column
**URL:** https://discuss.elastic.co/t/merging-csv-files-based-on-a-common-key-column/137868
**Category:** Logstash
**Created:** [June 28, 2018, 10:07pm UTC](https://discuss.elastic.co/t/merging-csv-files-based-on-a-common-key-column/137868 "2018-06-28T22:07:33Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Rakesh5216](https://discuss.elastic.co/u/Rakesh5216)
#### Post date: [June 28, 2018, 10:07pm UTC](https://discuss.elastic.co/t/merging-csv-files-based-on-a-common-key-column/137868/1 "2018-06-28T22:07:33Z")
Hello All,
I have been reading the documentation and various posts on elastic community, but still difficult to find solution to this problem. I have 2 csv files, I need to merge into single csv file and inject the data into elasticsearch. I have read through filter plugin , translate plugin but still i need help. I am stuck with same problem for 3 weeks. Below is my config file.
I need help on the filter part where i can use compare a particular column thats user\_id if user\_id matches in both csv files, then i need to put all data from both csv files into single csv file. Please help me out.
input {
file {
type =\> "transaction"
path =\> "C:\Users\Desktop\DATA\TRANSACTION\_TIME.csv "
start\_position =\> "beginning"
sincedb\_path =\> "/dev/null"
}
```
file {
type => "subscriber"
path => "C:\Users\Desktop\DATA\SUBSCRIBER.csv "
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
```
####### FILTER ####################################################
filter {
csv {
separator =\> ","
}
}
####### OUTPUT ###################################################
output {
elasticsearch {
hosts =\> "[http://localhost:9200](http://localhost:9200)"
index =\> "data\_insights"
document\_type =\> "data"
}
stdout {}
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 2, 2018, 5:13pm UTC](https://discuss.elastic.co/t/merging-csv-files-based-on-a-common-key-column/137868/2 "2018-07-02T17:13:19Z")
Suppose we have a CSV that looks like this, where the second field is the key.
```
some,foo,other,words
more,bar,stuff,here
```
We can use logstash to convert this to something that looks like
```
foo,"some,foo,other,words"
bar,"more,bar,stuff,here"
```
Using something like this:
```
input { file { path => "/path/to/foo.csv" start_position => "beginning" sincedb_path => "/dev/null" } }
filter { csv {} }
output { stdout { codec => plain { format => '%{column2},"%{message}"
' } } }
```
Then you can use that as the input to a translate filter
```
translate {
dictionary_path => "/path/to/another.csv"
field => "column1"
}
csv {
source => "translation"
columns => ["c1", "c2", "c3", "c4"]
}
```
If you have quoted fields in the first CSV file then you would have to gsub to something else in that first logstash and gsub them back in the second.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 30, 2018, 5:13pm UTC](https://discuss.elastic.co/t/merging-csv-files-based-on-a-common-key-column/137868/3 "2018-07-30T17:13:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.