# Fingerprint no consistent on combination of same type of value

**URL:** <https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038>\
**Category:** Logstash\
**Created:** [March 23, 2024, 2:54pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038 "2024-03-23T14:54:42Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![kishorkumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kishorkumar/32/132930_2.png) [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\
**Post date:** [March 23, 2024, 2:54pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/1 "2024-03-23T14:54:42Z")

</div>

Hello Everyone , i am trying to create a unique fingerprint figure print according to upc + displayName and upc + selectedModifiersCombined with the conditions  
Here are 4 orders after splitting in logstash  
{  
orderId: "1548"  
items:{  
"upc": "1236",  
"displayName": "Coffee"  
"selectedModifiersCombined": "Coffee Large Hot"}  
}  
{  
orderId: "1648"  
items:{  
"upc": "1236",  
"displayName": "Coffee",  
"selectedModifiersCombined": "Coffee Large Hot"}  
}{  
orderId: "16468"  
items:{  
"upc": "12386",  
"displayName": "Chicken Curry"  
}  
}{  
orderId: "17468"  
items:{  
"upc": "12386",  
"displayName": "Chicken Curry"  
}  
}I have used this code in order to create this but its create different fingerprints  
if [items][selectedModifiers] and [items][selectedModifiers][0] {  
fingerprint {  
source =\> ["[items][upc]", "[items][selectedModifiersCombined]"]  
target =\> "fingerprint"  
method =\> "SHA1"  
concatenate\_all\_fields =\> "true"  
}  
} else {  
fingerprint {  
source =\> ["[items][upc]", "[items][displayName]"]  
target =\> "fingerprint"  
method =\> "SHA1"  
concatenate\_all\_fields =\> "true"  
}  
}So why it's not generating same fingerprint for same type of itemshere is the one of the order can you guys help me with that [#stack-logstash](https://elasticstack.slack.com/archives/CNKF2D325) [#stack-elasticsearch]  
([https://elasticstack.slack.com/archives/CNTP6NLTZ](https://elasticstack.slack.com/archives/CNTP6NLTZ))

i have done it with ruby to generate same has and it is working but it does not make sense that why fingerprint is not doing

THis is the first data recorded Order in real test data

"items":[{"name":"657227000506","displayName":"Essentia Purified Water, Ionized Hydration 50.7 fl oz (1 qt 1 pt 2.7 fl oz) 1.5 l","qbName":"","type":"Beverages","subType":"Beverages","upc":"657227000506",

this is the second order in same test data

"items":[{"name":"657227000506","displayName":"Essentia Purified Water, Ionized Hydration 50.7 fl oz (1 qt 1 pt 2.7 fl oz) 1.5 l","qbName":"","type":"Beverages","subType":"Beverages","upc":"657227000506",

Here is the code

if [items][selectedModifiersCombined] {  
fingerprint {  
source =\> ["[items][upc]","[items][selectedModifiersCombined]"]  
target =\> "fingerprint"  
method =\> "SHA1"  
concatenate\_all\_fields =\> true  
}  
} else {  
fingerprint {  
source =\> ["[items][upc]","[items][displayName]"]  
target =\> "fingerprint"  
method =\> "SHA1"  
concatenate\_all\_fields =\> true  
}  
}

Here is the output of that using that code

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/2/c2f639c1f87b18ff7a97520d354ceee6fb8e1c53.png)

now i here is the ruby code that i have used it's working but i wonder why logstash fingerprint not working

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/f/af5d6727c698c0d63d2b089ff05c7eecbad8882e.png)

#Logstash #Elasticsearch

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 23, 2024, 4:05pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/2 "2024-03-23T16:05:29Z")

</div>

Hello,

Can you please format your pipeline using the preformatted text option? The `</>` button. Without it is hard to understand.

Also, where this `id` field comes from? You highlighted it on your screenshots, but it is not clear from where this `id` field com from and where this is used.

The values you highlighted are different.

Can you share your entire configuration to make it clear what it is doing? And also some sample messages so this can be replicated.

---

<div class="post-metadata">

**Author:** ![kishorkumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kishorkumar/32/132930_2.png) [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\
**Post date:** [March 23, 2024, 5:44pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/3 "2024-03-23T17:44:17Z")

</div>

> [@leandrojmp](#):
>
> ou highlighted it on your screensh

the highlighted id just to check that it's different order.

its not used in the fingerprint  
.

---

<div class="post-metadata">

**Author:** ![kishorkumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kishorkumar/32/132930_2.png) [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\
**Post date:** [March 23, 2024, 5:49pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/6 "2024-03-23T17:49:24Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/a/8/a8e6f36cfa2379668b9aaedb9b0e0422d3ae20c2.png)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 23, 2024, 5:57pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/7 "2024-03-23T17:57:56Z")

</div>

You need to share some samples messages so this can be replicated.

Share a complete sample message where the fingerprint is working and one where it is not working.

---

<div class="post-metadata">

**Author:** ![kishorkumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kishorkumar/32/132930_2.png) [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\
**Post date:** [March 23, 2024, 6:41pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/8 "2024-03-23T18:41:54Z")

</div>

```auto
{"userId":"33a4-1ff9-d8-a7fb-d1257","id":"48-0d-44e7-bc-b11","key":"5E74-1218-4F64-B7-947/+1:16:10.502Z/4811","items":[{"name":"657227000506","displayName":"Essentia Purified Water, Ionized Hydration 50.7 fl oz (1 qt 1 pt 2.7 fl oz) 1.5 l","qbName":"","type":"Beverages","subType":"Beverages","upc":"657227000506","modifiers":[],"hasModifiers":false,"selectedModifiers":[]},{"name":"doms-cst-upc-43515","displayName":"Asian Chicken Salad","qbName":"","type":"The Plant Butcher","subType":"Grab & Go Salads","upc":"43515","selectedModifiers":[{"name":"doms-add-utensil-option-yes","displayName":"Yes","price":0,"upc":"0002","modifierDisplayName":"Utensils","modifierIndex":0,"modifierType":"modifier"}],"hasModifiers":true},{"name":"04904500","displayName":"Coca Cola Diet Coke 20 oz","qbName":"","type":"Beverages","subType":"Beverages","upc":"04904500","hasModifiers":false,"selectedModifiers":[]},{"name":"doms-cst-upc-12474","displayName":"Fudge Brownie","qbName":"","type":"Desserts","subType":"Dessert Bars","upc":"12474","hasModifiers":false,"selectedModifiers":[]},{"name":"doms-cst-upc-43502","displayName":"Buffalo Chicken Salad","qbName":"","type":"The Plant Butcher","subType":"Made to Order Salads","upc":"43502","hasModifiers":true,"selectedModifiers":[{"name":"doms-add-utensil-option-yes","displayName":"Yes","price":0,"upc":"0002","modifierDisplayName":"Utensils","modifierIndex":0,"modifierType":"modifier"},{"name":"Dressingontheside-mod-option-0-1699898403266","displayName":"Dressing on the side","price":0,"upc":"MTO1554","modifierDisplayName":"Dressing on the side","modifierIndex":1,"modifierType":"modifier"},{"name":"NoOnion-mod-option-0-1700690395038","displayName":"No Onion","price":0,"upc":"MTO1550","modifierDisplayName":"Remove Ingredients","modifierIndex":3,"modifierType":"modifier"},{"name":"NoCherryTomatoes-mod-option-5-1700690395038","displayName":"No Cherry Tomatoes","price":0,"upc":"MTO1510","modifierDisplayName":"Remove Ingredients","modifierIndex":3,"modifierType":"product"}]}]}
{"userId":"3a4-1ff9-d8-a7fb-13w257","id":"418-0d-44e7-bc-b211","key":"5wE74-1218-4F64-B7-947/+1:16:10.502Z/4811","items":[{"name":"657227000506","displayName":"Essentia Purified Water, Ionized Hydration 50.7 fl oz (1 qt 1 pt 2.7 fl oz) 1.5 l","qbName":"","type":"Beverages","subType":"Beverages","upc":"657227000506","modifiers":[],"hasModifiers":false,"selectedModifiers":[]},{"name":"doms-cst-upc-43515","displayName":"Asian Chicken Salad","qbName":"","type":"The Plant Butcher","subType":"Grab & Go Salads","upc":"43515","selectedModifiers":[{"name":"doms-add-utensil-option-yes","displayName":"Yes","price":0,"upc":"0002","modifierDisplayName":"Utensils","modifierIndex":0,"modifierType":"modifier"}],"hasModifiers":true},{"name":"04904500","displayName":"Coca Cola Diet Coke 20 oz","qbName":"","type":"Beverages","subType":"Beverages","upc":"04904500","hasModifiers":false,"selectedModifiers":[]},{"name":"doms-cst-upc-12474","displayName":"Fudge Brownie","qbName":"","type":"Desserts","subType":"Dessert Bars","upc":"12474","hasModifiers":false,"selectedModifiers":[]},{"name":"doms-cst-upc-43502","displayName":"Buffalo Chicken Salad","qbName":"","type":"The Plant Butcher","subType":"Made to Order Salads","upc":"43502","hasModifiers":true,"selectedModifiers":[{"name":"doms-add-utensil-option-yes","displayName":"Yes","price":0,"upc":"0002","modifierDisplayName":"Utensils","modifierIndex":0,"modifierType":"modifier"},{"name":"Dressingontheside-mod-option-0-1699898403266","displayName":"Dressing on the side","price":0,"upc":"MTO1554","modifierDisplayName":"Dressing on the side","modifierIndex":1,"modifierType":"modifier"},{"name":"NoOnion-mod-option-0-1700690395038","displayName":"No Onion","price":0,"upc":"MTO1550","modifierDisplayName":"Remove Ingredients","modifierIndex":3,"modifierType":"modifier"},{"name":"NoCherryTomatoes-mod-option-5-1700690395038","displayName":"No Cherry Tomatoes","price":0,"upc":"MTO1510","modifierDisplayName":"Remove Ingredients","modifierIndex":3,"modifierType":"product"}]}]}

```

ndjson file

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 23, 2024, 6:59pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/9 "2024-03-23T18:59:29Z")

</div>

> [@leandrojmp](#):
>
> Can you share your entire configuration to make it clear what it is doing?

You didn't share this, it is not possible to know what can be your issue if you do not share how you are processing your logs.

For example, your logs have arrays, which fields are you splitting as individual events and which are you not splitting?

---

<div class="post-metadata">

**Author:** ![kishorkumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kishorkumar/32/132930_2.png) [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\
**Post date:** [March 23, 2024, 7:03pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/12 "2024-03-23T19:03:46Z")

</div>

> [@kishorkumar](#):
>
> ```auto
> filter {
> json{
> source => "message"
> }
> 
> # Removing extra fields
> mutate {
> remove_field => ["message", "event", "@version", "host", "path"]
> 
> }
> 
> # Splitting the items
> if [items] and [items][0]{
> split {
> field => "[items]"
> }
> }
>  
> 
> fingerprint {
> source => ["[items][upc]","[items][displayName]"]
> target => "fingerprint"
> method => "SHA1"
> concatenate_all_fields => true
> }
>   
> 
> }
> 
> ```

the file of filter is give below i think you know how the input works

---

<div class="post-metadata">

**Author:** ![kishorkumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kishorkumar/32/132930_2.png) [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\
**Post date:** [March 23, 2024, 7:12pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/14 "2024-03-23T19:12:28Z")

</div>

i have simplified the work more so don't confuse in if else

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 23, 2024, 7:14pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/15 "2024-03-23T19:14:52Z")

</div>

You have a wrong setting:

```auto
concatenate_all_fields => true

```

Using `concatenate_all_fields` will make logstash concatenate **all** fields in the event before doing the fingerprint, and since you have different fields, the fingerprint will be different.

What you need to use is `concatenate_sources`, this will make logstash concatenate **only** the fields that are specified in the `source` option, in this case only `[items][upc]` and `[items][displayName]`.

Replace `concatenate_all_fields => true` with `concatenate_sources => true` and it should work, as it worked for me.

With `concatenate_sources` set as `true`

```auto
$ grep "Essentia Purified" output-discuss.log | jq .fingerprint
"278fc6caf37c4b923a60741f4f16b37816c91dc2"
"278fc6caf37c4b923a60741f4f16b37816c91dc2"

```

And with `concatenate_all_fields` set as `true`.

```auto
$ grep "Essentia Purified" output-discuss.log | jq .fingerprint
"de543b5ec672b4efa9185a8b53ba5345677137db"
"daefbc9d686489e101f397fb599f13de68dcd615"

```

---

<div class="post-metadata">

**Author:** ![kishorkumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kishorkumar/32/132930_2.png) [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\
**Post date:** [March 23, 2024, 7:23pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/16 "2024-03-23T19:23:09Z")

</div>

> [@leandrojmp](#):
>
> concatenate\_sources

Thanks mate for clarifying the working of `concatenate_sources` and `concatenate_all_fields`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 20, 2024, 7:23pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038/17 "2024-04-20T19:23:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.