Logstash join two csv files, one to many data

Elastic Stack Logstash
1

Hi,

I was able to follow instructions on joining 2 csv files from this link

However, this is just for joining one to one data, e.g.
file1.csv
id,name
1,john
2,jonathan

file2.csv
id,lastname
1,wick
2,smith

this results to
1,john,wick
2,jonathan,smith

I would like to know if there's a way we join 2 csv file with one to many data, e.g.
file1.csv
id,name
1,john
2,jonathan

file2.csv
id,lastname
1,wick
1,summers
2,smith
2,winters

which will result to
1,john,wick
1,john,summers
2, jonathan, smith
2, jonathan, winters

2

Use file1.csv as the dictionary in a translate filter. logstash does not care that there are many events that lookup the same entry.

3

Can an id ever have two entries in file1, eg:

1 John
1 Jerry
2 Alice

?

4

Under the hood translate is using a hash, so when it loads the file it will not object, but only Jerry will be included in the translation of 1.

5

I meant to direct my question to @genice_ladroma :slight_smile:

If file1 doesn't have unique id's, then the dictionary approach will not work.

6

Agreed, many to many will not work, but one to many should.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.