my csv file =>
name,surname,age,email,phone
Harry,Potter,18,NULL,NULL
Harry,Potter,NULL,harrypotter@gmail.com,+955555555
Harry,Potter,NULL,harrypotter@gmail.com,NULL
Harry,Potter,NULL,NULL,+955555555
When I want to detect and delete duplicate documents with
fingerprint method, it creates a new document for each row.
filter {
fingerprint {
key => "1234ABCD"
method => "MD5"
source => ["name","surname","age","email","phone"]
target => "[@metadata][generated_id]"
}
}
output {
stdout { codec => dots }
elasticsearch {
index => "null_problem_fingerprint"
document_id => "%{[@metadata][generated_id]}"
action => 'update'
}
}
If I specify only the name and surname fields for the source as in the code blog below
this time it does not read the other rows after reading the first row.
filter {
fingerprint {
key => "1234ABCD"
method => "MD5"
source => ["name","surname"]
target => "[@metadata][generated_id]"
}
}
output {
stdout { codec => dots }
elasticsearch {
index => "null_problem_fingerprint"
document_id => "%{[@metadata][generated_id]}"
action => 'update'
}
}
dear friends please help me! İ want to see just one document like this;
Harry,Potter,18,harrypotter@gmail.com,+955555555