# "uuid" and "fingerprint" fields

**URL:** <https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794>\
**Category:** Logstash\
**Created:** [August 24, 2016, 9:37am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794 "2016-08-24T09:37:49Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![gayathri](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@gayathri](https://discuss.elastic.co/u/gayathri)\
**Post date:** [August 24, 2016, 9:37am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/1 "2016-08-24T09:37:50Z")

</div>

Hi Team,

I am using below configuration in logstash.conf for avoiding duplicate log entries:  
filter{  
uuid {  
target =\> "@uuid"  
overwrite =\> true  
}  
fingerprint {  
source =\> ["message"]  
target =\> "fingerprint"  
key =\> "78787878"  
method =\> "SHA1"  
concatenate\_sources =\> true  
target =\> "[@metadata][fingerprint]"  
}

}}  
output {  
elasticsearch {  
host =\> localhost  
document\_id =\> "%{[@metadata][fingerprint]}"  
}  
stdout { codec =\> rubydebug }  
}

Is there any issues on using this method.?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 24, 2016, 11:04am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/2 "2016-08-24T11:04:48Z")

</div>

I don't remember if you need to add an extra option to the ES output to get upsert behavior. Apart from that possible issue this should be fine except. What's the purpose of the `uuid` field?

---

<div class="post-metadata">

**Author:** ![gayathri](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@gayathri](https://discuss.elastic.co/u/gayathri)\
**Post date:** [August 24, 2016, 11:05am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/3 "2016-08-24T11:05:52Z")

</div>

inorder to remove the duplication

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 24, 2016, 11:09am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/4 "2016-08-24T11:09:03Z")

</div>

But

- UUIDs are random and therefore useless for duplicate elimination, and
- you're not using the `uuid` field anywhere in the rest of your configuration.

---

<div class="post-metadata">

**Author:** ![gayathri](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@gayathri](https://discuss.elastic.co/u/gayathri)\
**Post date:** [August 24, 2016, 11:20am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/5 "2016-08-24T11:20:15Z")

</div>

so you mean "fingerprint" field is enough for removing the duplication?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 24, 2016, 11:32am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/6 "2016-08-24T11:32:52Z")

</div>

Yes.

---

<div class="post-metadata">

**Author:** ![gayathri](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@gayathri](https://discuss.elastic.co/u/gayathri)\
**Post date:** [August 24, 2016, 11:34am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/7 "2016-08-24T11:34:05Z")

</div>

ohk....will it impact anything except duplication ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 24, 2016, 11:35am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/8 "2016-08-24T11:35:04Z")

</div>

Will _what_ impact anything? The mere presence of the `[@metadata][fingerprint]` field?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:41am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794/9 "2017-07-06T04:41:52Z")

</div>