Append field if document_id already exists

Elastic Stack Logstash
1

I have a csv file with multiple records for each document where the data is partially different.

doc_id insurance_policy policy_holder
C12 P-C12-2017 Michal
C12 P-C12-2017 Sophie
C12 P-C12-2017 Adam
C12 P-C12-2017 Peter

I am processing the above csv file using the csv plugin in Logstash to insert data into elastic.
Instead of having 4 documents in elastic, my requirement is to have only one document with all data in it.

Intended document in elastic
{
_id: C12,
policy: P-C12-2017,
holders: [ Michal, Sophie, Adam, Peter]
}

How can I achieve this using Logstash?

(I tried using elasticsearch plugin to lookup already existing document to fetch the policy_holder data but this only picks up one entry)

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.