# Real time and sync

**URL:** <https://discuss.elastic.co/t/real-time-and-sync/237859>\
**Category:** Elasticsearch\
**Created:** [June 19, 2020, 7:54pm UTC](https://discuss.elastic.co/t/real-time-and-sync/237859 "2020-06-19T19:54:01Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![asl](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@asl](https://discuss.elastic.co/u/asl)\
**Post date:** [June 19, 2020, 7:54pm UTC](https://discuss.elastic.co/t/real-time-and-sync/237859/1 "2020-06-19T19:54:01Z")

</div>

Hi everyone, I’m trying to achieve real-time operations without affecting performance too much.  
More in deep, I need to sync data from a main relational database to ES. I thought about using a message broker (or by using an ETL and by scanning it every X minutes to perform same operations on ES).

I faced the issue regarding ES near real-time.  
In a scenario with just clients and ES indexes (without a sync with a RDBS): let’s suppose a refresh occurs at t2.

1. A new document is indexed at t0
2. The same document is deleted (or updated) at t1.

This would ends up with a non-existing document. This could be solved by accepting the ES near real-time behavior and let clients to get a not found result or by using a forced refresh before performing 1) or 2).

**Question 1** : If inserts are way more frequent (and in row) than deletes and updates, is it better to refresh before deletes/updates?

**Question 2** : If accepting a not found result would be feasible with just clients \<—\> ES, what about clients \<—\> RDBS \<—\> ES ?  
The document would be correctly deleted from RDBS (which is real-time) and the same 2 operations would be enqueued using a message queue and will be get executed **in row at unknown time** on ES. This would always fail without an explicit refresh and there will be no consistency between RDBS and ES because the document will be indexed on ES. There wouldn’t be a way to let clients noticing about the fail because those two operations could have been performed a lot time before on RDBS.

I’ve also read [https://www.elastic.co/blog/found-keeping-elasticsearch-in-sync#using-queues-to-manage-batches](https://www.elastic.co/blog/found-keeping-elasticsearch-in-sync#using-queues-to-manage-batches) but it looks like it doesn’t face the issue.

The only way I found is just triggering an explicit refresh before deleting or updating a document

I hope to have explained my problem as much clear as possible

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 17, 2020, 8:07pm UTC](https://discuss.elastic.co/t/real-time-and-sync/237859/2 "2020-07-17T20:07:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.