# Documents not getting udpated as per the pipeline

**URL:** <https://discuss.elastic.co/t/documents-not-getting-udpated-as-per-the-pipeline/309394>\
**Category:** Logstash\
**Created:** [July 12, 2022, 9:29am UTC](https://discuss.elastic.co/t/documents-not-getting-udpated-as-per-the-pipeline/309394 "2022-07-12T09:29:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Apoorv\_Agarwal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/apoorv_agarwal/32/108189_2.png) [@Apoorv\_Agarwal](https://discuss.elastic.co/u/Apoorv_Agarwal)\
**Post date:** [July 12, 2022, 9:29am UTC](https://discuss.elastic.co/t/documents-not-getting-udpated-as-per-the-pipeline/309394/1 "2022-07-12T09:29:03Z")

</div>

I am using one elapsed plugin to calculate the time difference between two events, with the output sending data into a "merged" index. For some reason, at certain times, the merged index is not getting updated in real time, and because of the delay, wrong alerts are being triggered.  
I cant seem to figure out the reason behind the same, as its happening quite randomly.

I am sharing the **filter** section of my pipeline below, the input contains data ingestion from 3 kafka topics, and the **output** section is sending data to ES, with every document being updated against a fingerprint.

```auto
filter {
    json {
        source => "message"
    }

    fingerprint {
        source => ["[data][args][conversationId]"]
    }
    
    if [data][args][chatData][source] == "zd:agentWorkspace" {
        mutate {
            add_tag => ["taskStarted"]
        }
    }
    if [data][args][chatData][source] == "web" {
        mutate {
            add_tag => ["taskTerminated"]
        }
    }

    elapsed {
        start_tag => "taskStarted"
        end_tag => "taskTerminated"
        unique_id_field => "fingerprint"
        timeout => 190
        keep_start_event => "last"
    }
    if "elapsed_expired_error" in [tags] {
        mutate {
            remove_field => ["[host]" ]
        }
    }
}

```

This is a trimmed version of the filter section. I have removed the mutate and elasticsearch filter plugins in there.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 9, 2022, 9:29am UTC](https://discuss.elastic.co/t/documents-not-getting-udpated-as-per-the-pipeline/309394/2 "2022-08-09T09:29:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.