# Fingerprint plugin not working properly

**URL:** <https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845>\
**Category:** Logstash\
**Created:** [April 5, 2024, 8:01am UTC](https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845 "2024-04-05T08:01:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Amer\_Uljic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amer_uljic/32/121946_2.png) [@Amer\_Uljic](https://discuss.elastic.co/u/Amer_Uljic)\
**Post date:** [April 5, 2024, 8:01am UTC](https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845/1 "2024-04-05T08:01:12Z")

</div>

I have logs that I receive daily in a JSON form and most of them contain unique identifiers such as ip addresses and ports. A lot of them repeat daily so I want to filter them out but their timestamp changes and elastic creates a new id for every single one of them. I have installed and configured fingerprint plugin in logstash to create a unique fingerprint for every event based on that events ip address and port. For some reason logstash creates the same hash for every single event and not based on that events ip address and port.  
I expect results such as:  
192.168.0.1:80 = n3j1tjo31ifj0inv023n10f  
192.168.0.2:443=31tboginv4go1ng4igno4

But I receive  
192.168.0.1:80 = n3j1tjo31ifj0inv023n10f  
192.168.0.2:443=n3j1tjo31ifj0inv023n10f

I will include the logstash.conf below. Thank you for any assistance in advance.

```auto
input {
   file {
   codec => "json"
   path => "/usr/share/shadow/one.json" 
   start_position => beginning 
   sincedb_path => "/dev/null"
   #sincedb_path => "/home/bitnami/sincedb/sincedb-access"
  }
}
filter {
 fingerprint{
                source => ["ip","port"]
                concatenate_sources => true
                target => "jedinstveni_id"
                method => "MD5"
                key => "randomkey"
                }
}
output {
  elasticsearch {
    hosts => ["127.0.0.1:9200"]
    data_stream => false
    #document_id => "%{logstash_checksum}"
    index => "shadow-server"
  }
  stdout {
    codec => rubydebug
  }
  file {
    path => "/usr/share/shadow/log/output.log"
  }
}

```

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 5, 2024, 9:16am UTC](https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845/2 "2024-04-05T09:16:34Z")

</div>

Welcome to the community!

Not sure how do you get the same hash. The hash should be doc\_id. No need for a random key, since you want only deduplication.

This sample is working:

```auto
input {
  generator {
       message => '{"ip": "192.168.0.1","port":"80"}
	   {"ip": "192.168.0.1", "port":"43"}'
	   count => 1
	   codec => json_lines
  }
}

filter {

   fingerprint{
                source => ["ip","port"]
                concatenate_sources => true
                target => "[@metadata][jedinstveni_id]"
                method => "MD5"
                #key => "randomkey"
   }
  
   mutate { remove_field => ["host", "event", "@timestamp", "@version"] } 

}

output {
  stdout { codec => rubydebug{ metadata => true}}
  elasticsearch {
    hosts => ["127.0.0.1:9200"]
    data_stream => false
    document_id => "%{[@metadata][jedinstveni_id]}"
    index => "shadow-server"
  }
}

```

Result:

```auto
{
         "port" => "80",
    "@metadata" => {
        "jedinstveni_id" => "4f1bead294d01181b6c40ac78deb5381"
    },
           "ip" => "192.168.0.1"
}
{
         "port" => "43",
    "@metadata" => {
        "jedinstveni_id" => "ef7a9233096432e9a67f72931aac7155"
    },
           "ip" => "192.168.0.1"
}

```

---

<div class="post-metadata">

**Author:** ![Amer\_Uljic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amer_uljic/32/121946_2.png) [@Amer\_Uljic](https://discuss.elastic.co/u/Amer_Uljic)\
**Post date:** [April 5, 2024, 9:52am UTC](https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845/3 "2024-04-05T09:52:11Z")

</div>

Thank you for your assistance but the config is still giving me the same hash values across the events. Do you think the JSON file I'm reading from could be at fault?

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 5, 2024, 10:10am UTC](https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845/4 "2024-04-05T10:10:17Z")

</div>

Might be, you should review with your data, just put in the generator.

---

<div class="post-metadata">

**Author:** ![Amer\_Uljic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amer_uljic/32/121946_2.png) [@Amer\_Uljic](https://discuss.elastic.co/u/Amer_Uljic)\
**Post date:** [April 5, 2024, 10:32am UTC](https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845/5 "2024-04-05T10:32:29Z")

</div>

It works with the generator, thank you! So my issue is probably with the json file or the way logstash interprets it.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 5, 2024, 11:59am UTC](https://discuss.elastic.co/t/fingerprint-plugin-not-working-properly/356845/6 "2024-04-05T11:59:18Z")

</div>

I don't know is it a single or a multiline JSON. If is the multiline file- the human readable with spaces&new lines, cannot be converted in the regular JSON codec.