# Logstash API,Logstash doesn't update the data when mongodb data gets updated or delete records

**URL:** <https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183>\
**Category:** Logstash\
**Tags:** license\
**Created:** [April 25, 2024, 9:03am UTC](https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183 "2024-04-25T09:03:11Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![anguri\_sudhakar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anguri_sudhakar/32/50050_2.png) [@anguri\_sudhakar](https://discuss.elastic.co/u/anguri_sudhakar)\
**Post date:** [April 25, 2024, 9:03am UTC](https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183/1 "2024-04-25T09:03:11Z")

</div>

As records are updated, modified and deleted in our MongoDB transactional database, those documents Logstsah API will not fetching the records in ES

**logstash: 8.5.1**  
**Elasticsearch:8.9.1**

Is there an option? What would be the best way to use MongoDB, Logstash and Elastic?

---

<div class="post-metadata">

**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\
**Post date:** [April 25, 2024, 9:10am UTC](https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183/2 "2024-04-25T09:10:09Z")

</div>

Hi @anguri_sudhakar,

Welcome back. How are you currently pulling the records from MongoDB to Elasticsearch? Are you using the [JDBC input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html)? Or an alternative approach.

---

<div class="post-metadata">

**Author:** ![anguri\_sudhakar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anguri_sudhakar/32/50050_2.png) [@anguri\_sudhakar](https://discuss.elastic.co/u/anguri_sudhakar)\
**Post date:** [April 25, 2024, 9:17am UTC](https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183/3 "2024-04-25T09:17:28Z")

</div>

Thanks for quick response 🙂 @carly.richmond

Here is my config file:  
input {  
mongodb {  
codec =\> "json"  
uri =\> 'mongodb://mongo-admin:mongo\_admin'  
placeholder\_db\_dir =\> '/data/logstash-mongodb/'  
placeholder\_db\_name =\> 'xx\_elk\_dev.db'  
collection =\> 'elk\_xxx\_data'  
batch\_size =\> 50000  
}  
}

filter {  
mutate {  
#remove\_field =\> ["\_id"]  
rename =\> { "\_id" =\> "mongo\_id" }  
}  
date {  
match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]  
}  
}

output {  
elasticsearch {

```
      hosts => ["http://xx.xx.xx.xx:9200"]
           user => "elastic"
           password => "xxxx"
           #index => "employee-%{+YYYY.MM.dd}"
           index => "elk_xxx_data"
           document_id => "%{_id}"

```

}  
stdout { codec =\> rubydebug }  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 25, 2024, 4:37pm UTC](https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183/4 "2024-04-25T16:37:40Z")

</div>

> [@anguri\_sudhakar](#):
>
> As records are updated, modified and deleted in our MongoDB transactional database, those documents Logstsah API will not fetching the records in ES

That is correct. The mongodb input [looks for items](https://github.com/phutchins/logstash-input-mongodb/blob/097cc9be8bb57754ec4006e174f0cb5cf3ff6b65/lib/logstash/inputs/mongodb.rb#L151) with an \_id value larger than the last \_id it retrieved. The \_id field is the immutable identifier of a document, so if the document is updated then the \_id does not change. The mongodb input will never see it. Similarly for deletions.

You may be able to implement the tracking of updates in mongodb using [change streams](https://www.mongodb.com/docs/manual/changeStreams/), but that would be outside of logstash.

---

<div class="post-metadata">

**Author:** ![anguri\_sudhakar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anguri_sudhakar/32/50050_2.png) [@anguri\_sudhakar](https://discuss.elastic.co/u/anguri_sudhakar)\
**Post date:** [April 26, 2024, 12:54pm UTC](https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183/5 "2024-04-26T12:54:05Z")

</div>

Thanks for update @carly.richmond @Badger

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 24, 2024, 12:54pm UTC](https://discuss.elastic.co/t/logstash-api-logstash-doesnt-update-the-data-when-mongodb-data-gets-updated-or-delete-records/358183/6 "2024-05-24T12:54:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.