[Threat Intelligence]: Avoid redundancy of information in the same index

Elastic Stack Logstash
4

To generate a fingerprint use a filter

fingerprint {
    concatenate_sources => true 
    method => "SHA256" 
    source => [ "url" ] # And possibly other fields
    target => "[@metadata][fingerprint]"
}

then reference it in the elasticsearch output using a sprintf reference

document_id => "%{[@metadata][fingerprint]}"
1 Like