Logstash join two csv files, one to many data

Hi,

I was able to follow instructions on joining 2 csv files from this link

However, this is just for joining one to one data, e.g.
file1.csv
id,name
1,john
2,jonathan

file2.csv
id,lastname
1,wick
2,smith

this results to
1,john,wick
2,jonathan,smith

I would like to know if there's a way we join 2 csv file with one to many data, e.g.
file1.csv
id,name
1,john
2,jonathan

file2.csv
id,lastname
1,wick
1,summers
2,smith
2,winters

which will result to
1,john,wick
1,john,summers
2, jonathan, smith
2, jonathan, winters

Use file1.csv as the dictionary in a translate filter. logstash does not care that there are many events that lookup the same entry.

Can an id ever have two entries in file1, eg:

1 John
1 Jerry
2 Alice

?

Under the hood translate is using a hash, so when it loads the file it will not object, but only Jerry will be included in the translation of 1.

I meant to direct my question to @genice_ladroma :slight_smile:

If file1 doesn't have unique id's, then the dictionary approach will not work.

Agreed, many to many will not work, but one to many should.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.