# Logstash fingerprint plugin with JSON data

**URL:** <https://discuss.elastic.co/t/logstash-fingerprint-plugin-with-json-data/147239>\
**Category:** Logstash\
**Created:** [September 4, 2018, 3:25pm UTC](https://discuss.elastic.co/t/logstash-fingerprint-plugin-with-json-data/147239 "2018-09-04T15:25:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![krishna\_chaitanya](https://avatars.discourse-cdn.com/v4/letter/k/b5a626/32.png) [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\
**Post date:** [September 4, 2018, 3:25pm UTC](https://discuss.elastic.co/t/logstash-fingerprint-plugin-with-json-data/147239/1 "2018-09-04T15:25:26Z")

</div>

Hi,

I am parsing the JSON logs using below config:

```
  file {
        type =>"logstash"
        path => ["/logstash/*.json"]
        sincedb_path =>"/logstash/var/json.sincedb"
        codec => json
        start_position => "beginning"
  }

```

Later in the pipeline, I would like to use fingerprint plugin in order to avoid duplicates.  
Something like this:

```
filter {
  fingerprint {
    source => "message"
    target => "[@metadata][fingerprint]"
    method => "MURMUR3"
  }
}
output {
  elasticsearch {
    hosts => "example.com"
    document_id => "%{[@metadata][fingerprint]}"
  }
}

```

But, `message` field is not available since the logs are being parsed using json codec.  
How can I use fingerprint plugin on entire json event (all json fields) ?

Please advise. Thanks !!

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 4, 2018, 3:54pm UTC](https://discuss.elastic.co/t/logstash-fingerprint-plugin-with-json-data/147239/2 "2018-09-04T15:54:57Z")

</div>

Instead of using the JSON codec, you could read it in as a string and then run `fingerprint` on the text before using a json filter to parse the data. I would also recommend using a longer hash, e.g. SHA1 instead of MURMUR in order to reduce the hash collision probability.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 2, 2018, 3:55pm UTC](https://discuss.elastic.co/t/logstash-fingerprint-plugin-with-json-data/147239/3 "2018-10-02T15:55:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.