Hi,
I am parsing the JSON logs using below config:
file {
type =>"logstash"
path => ["/logstash/*.json"]
sincedb_path =>"/logstash/var/json.sincedb"
codec => json
start_position => "beginning"
}
Later in the pipeline, I would like to use fingerprint plugin in order to avoid duplicates.
Something like this:
filter {
fingerprint {
source => "message"
target => "[@metadata][fingerprint]"
method => "MURMUR3"
}
}
output {
elasticsearch {
hosts => "example.com"
document_id => "%{[@metadata][fingerprint]}"
}
}
But, message
field is not available since the logs are being parsed using json codec.
How can I use fingerprint plugin on entire json event (all json fields) ?
Please advise. Thanks !!