Join two documents

Elastic Stack Logstash
1

Hi,

I have two .csv files. I have a column in each document that is key for me ElasticSearch. When logstash indexes data overwrites the oldest values. Is it possible to keep these values in a table?
For example :

Csv file 1:
key; string
1; "abc"
2; "def"
3; "ghi"
2; "jkl"

Csv file 2:
key; number
1; 50
2, 47

ElasticSearch file:
key, string, number
1; "abc"; 50
2; "jkl"; 47
3; "ghi"

What I want:
1; [ "abc"]; [50]
2; ["def," jkl "]; [47]
3; [ "ghi"]

2

The documentation for elasticsearch filter plugin [1] gives a rough idea how to accomplish this. You just need to work out the ruby script filter to manage the merging.

[1] https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html

1 Like