# Should insert to new index or update existed documents with large data in ElasticSearch?

**URL:** <https://discuss.elastic.co/t/should-insert-to-new-index-or-update-existed-documents-with-large-data-in-elasticsearch/231280>\
**Category:** Elasticsearch\
**Created:** [May 6, 2020, 6:56am UTC](https://discuss.elastic.co/t/should-insert-to-new-index-or-update-existed-documents-with-large-data-in-elasticsearch/231280 "2020-05-06T06:56:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hi\_u\_Nguy\_n](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hi_u_nguy_n/32/65457_2.png) [@Hi\_u\_Nguy\_n](https://discuss.elastic.co/u/Hi_u_Nguy_n)\
**Post date:** [May 6, 2020, 6:56am UTC](https://discuss.elastic.co/t/should-insert-to-new-index-or-update-existed-documents-with-large-data-in-elasticsearch/231280/1 "2020-05-06T06:56:03Z")

</div>

Hello,  
I have a task,

1. Read very large log from agent send to elasticsearch (i use fluent-bit) in real-time
2. But, I need modify each record to add my tags by specific condition on each documents (function `add_tags(doc)` wrote by Python), So I need scheduler a job do:

- Get all documents not yet tagged from Elasticsearch
- Run throught the method `add_tags` to add field tags to document
- Put again to ElasticSearch

In last step, i need put again to ES, so, if I updated existed documents, is bad perfomance ?  
Or should to insert to new index ?

Thanks all.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 3, 2020, 6:56am UTC](https://discuss.elastic.co/t/should-insert-to-new-index-or-update-existed-documents-with-large-data-in-elasticsearch/231280/2 "2020-06-03T06:56:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.