# Fingerprint issue

**URL:** https://discuss.elastic.co/t/fingerprint-issue/25554
**Category:** Logstash
**Created:** [July 14, 2015, 8:43pm UTC](https://discuss.elastic.co/t/fingerprint-issue/25554 "2015-07-14T20:43:36Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Khoa\_Nguyen1](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@Khoa\_Nguyen1](https://discuss.elastic.co/u/Khoa_Nguyen1)
#### Post date: [July 14, 2015, 8:43pm UTC](https://discuss.elastic.co/t/fingerprint-issue/25554/1 "2015-07-14T20:43:36Z")

</div>

My CSV file has 26041 unique lines (I checked by manually generating sha1sum for each line). However, when using fingerprint as the document\_id, elastic only stores 25919 documents. One document has the wrong \_id:

```
_id: %{fingerprint} _type:logs _index:logstash-2015.07.14

```

This may have caused 122 records failed to load. Below is an excerpt of my config:

```
input { stdin {} }

filter {
    csv { ... }
    fingerprint { method => "SHA1" key => "RoamMonitor" }
}

output {
    elasticsearch {
        host => localhost
        cluster => "rm_cluster_dev"
        document_id => "%{fingerprint}"
    }
}

```

If I disable the fingerprint and use the auto\_id, all 26041 records were loaded.

Any ideas on how to debug/troubleshoot this?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 26, 2015, 10:41am UTC](https://discuss.elastic.co/t/fingerprint-issue/25554/2 "2015-07-26T10:41:31Z")

</div>

Calculate the fingerprint as you do now, but insert into Elasticsearch using auto\_id. You should then be able to identify the entries that result in duplicates, which should allow you to troubleshoot the issue.

---

<div class="post-metadata">

### Author: ![Khoa\_Nguyen1](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@Khoa\_Nguyen1](https://discuss.elastic.co/u/Khoa_Nguyen1)
#### Post date: [September 3, 2015, 6:19pm UTC](https://discuss.elastic.co/t/fingerprint-issue/25554/3 "2015-09-03T18:19:18Z")

</div>

I've been busy, but finally can get back to this. The issue was my configuration -- it turned out that I have some if statements that mess things up. Below is a simplified version of my config file:

```
input { stdin {} }

filter {
    if [message] =~ /^RECORD_A.*/ {
        fingerprint { 
            method => "SHA1"
            key => "mykey"
       }
    }
}

output { 
    elasticsearch { document_id => "%{fingerprint}" }
}

```

So if input record is **NOT** RECORD\_A, the fingerprint is not computed, and document\_id will take the string "%{fingerprint}" verbatim.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:30am UTC](https://discuss.elastic.co/t/fingerprint-issue/25554/4 "2017-07-06T05:30:12Z")

</div>