# Delta load in Elasticsearch

**URL:** https://discuss.elastic.co/t/delta-load-in-elasticsearch/310947
**Category:** Elasticsearch
**Created:** [July 29, 2022, 6:14am UTC](https://discuss.elastic.co/t/delta-load-in-elasticsearch/310947 "2022-07-29T06:14:18Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![madhanvs](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@madhanvs](https://discuss.elastic.co/u/madhanvs)
#### Post date: [July 29, 2022, 6:14am UTC](https://discuss.elastic.co/t/delta-load-in-elasticsearch/310947/1 "2022-07-29T06:14:18Z")

</div>

Hi  
I have two Elasticsearch indexes say Index\_A and Index\_B. Index\_A gets daily updates and inserts.  
I need to do a delta loading on Index\_B based on the data in Index\_A on a weekly basis.  
For Docy\_Type =A, I should consider Modified Date and for Docy\_Type =B, Created date should be considered.

My current indexes state:

![image](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4e2eae651ee5d0075530682652830e6ddeb2a1a8.png)

![image](https://us1.discourse-cdn.com/elastic/original/3X/f/f/ff0c9a22baa059571a38375af89cbc8b58a1b340.png)

So, after delta refresh, following conditions should be considered:

1. DocID 1 no update should happen in Index\_B as there no change in Modified Date.
2. DocID 2 should be deleted from Index\_B and new record from Index\_A should be inserted as there is a change in Modified Date.
3. DocID 3 should be deleted from Index\_B and new record from Index\_A should be inserted as there is a change in Created Date.
4. DocID 4 no update should happen in Index\_B as there no change in Created Date.
5. DocID 5 no update should happen in Index\_B as there no change in Created Date.
6. DocID 6 should be inserted in Index\_B as it is new/
7. DocID 7 should be deleted from Index\_B as it is obselete.

So, my Index\_B after Delta refresh:

![image](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d2309b00825acfed19bed0d569e47827cf24ecce.png)

Right now, my approach is to use Python scripting to achieve this.

Can anyone from Elasticsearch team shed some light how such delta refresh can be achieved this in optimal way? Is there any ES tool available?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 26, 2022, 6:14am UTC](https://discuss.elastic.co/t/delta-load-in-elasticsearch/310947/2 "2022-08-26T06:14:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.