# Filebeat and updating existing docs

**URL:** <https://discuss.elastic.co/t/filebeat-and-updating-existing-docs/320781>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 8, 2022, 12:04pm UTC](https://discuss.elastic.co/t/filebeat-and-updating-existing-docs/320781 "2022-12-08T12:04:37Z")\
**Posts on this page:** 1\
**Showing post:** 11

<div class="post-metadata">

**Author:** ![Marcin\_Frankiewicz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcin_frankiewicz/32/97650_2.png) [@Marcin\_Frankiewicz](https://discuss.elastic.co/u/Marcin_Frankiewicz)\
**Post date:** [December 13, 2022, 12:06pm UTC](https://discuss.elastic.co/t/filebeat-and-updating-existing-docs/320781/11 "2022-12-13T12:06:22Z")

</div>

I'm trying to update existing documents in elasticsearch, based on @metadata.\_id

Example

First published event

```auto
filebeat[26461]: DEBUG [processors] processing/processors.go:203 Publish event: {
                                                "@timestamp": "2022-12-10T10:10:28.695Z",
                                                "@metadata": {
                                                  "beat": "filebeat",
                                                  "type": "_doc",
                                                  "version": "7.17.5",
                                                  "_id": " **791f529272c4e95fbba6273ace4c3e6db0b2944e**"
                                                },
                                                "app": "app1",
                                                "test": {
                                                  "machine": {
                                                    "name": "1",
                                                    "description": "Test machine description2"
                                                  },
                                                  "prefix": "118fa800"
                                                }
                                              }

```

That document should be created in elasticsearch, beacuse doc with id : 791f529272c4e95fbba6273ace4c3e6db0b2944e, does not exists

After some time... filebeat crawls new file, with **new** test.machine.description

```auto
filebeat[26461]: DEBUG [processors] processing/processors.go:203 Publish event: {
                                                "@timestamp": "2022-12-12T10:10:28.695Z",
                                                "@metadata": {
                                                  "beat": "filebeat",
                                                  "type": "_doc",
                                                  "version": "7.17.5",
                                                  "_id": " **791f529272c4e95fbba6273ace4c3e6db0b2944e**"
                                                },
                                                "app": "app1",
                                                "test": {
                                                  "prefix": "118fa800",
                                                  "machine": {
                                                    "name": "1",
                                                    "description": " **New test machine description2**"
                                                  }

```

ID is the same like in previous event, so document should be **updated** (Thats what i'm trying to accomplish)

Unfortunately when i'm setting

```auto
    - fingerprint:
        fields: ["test.machine.name", "test.prefix"]
        target_field: "@metadata._id" -- **That line is problematic**
        method: "sha1"

```

doc never appear in kibana.. or i don't know where i can find it

This topic is similar to :

> [@Filebeat and updating documents](https://discuss.elastic.co/t/filebeat-and-updating-documents/167355):
>
> I am using Filebeat to bring database information into Elasticsearch. In some instances the existing table data (that has already been imported into Elasticsearch) is updated. How can I use Filebeat to update this data within Elasticsearch?

> **[Little Logstash Lessons: Handling Duplicates](https://www.elastic.co/blog/logstash-lessons-handling-duplicates)**
>
> Approaches for de-duplicating data in Elasticsearch using Logstash. We also go into examples of how you can use IDs in Elasticsearch Output.

> **[Deduplicate data | Filebeat Reference \[8.5\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/8.5/filebeat-deduplication.html)**

> [@Filebeat deduplication fail to update index](https://discuss.elastic.co/t/filebeat-deduplication-fail-to-update-index/231870):
>
> Hi, I'm reading this guide: [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-deduplication.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-deduplication.html), that discusses the deduplication in both filebeat and logstash, and recommends to set the document id before sending it to elasticsearch, this way the document get updated in case it already exists. But I've tried this, and I used Filebeat fingerprint processor, filebeat fails to update the document in elasticsearch if it already have the same id, while logstash does update the docum…

---

_[View the full topic](https://discuss.elastic.co/t/filebeat-and-updating-existing-docs/320781)._