# Data synchronization time in elasticsearch

**URL:** <https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748>\
**Category:** Elasticsearch\
**Created:** [August 28, 2020, 11:02am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748 "2020-08-28T11:02:16Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [August 28, 2020, 11:02am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/1 "2020-08-28T11:02:16Z")

</div>

Hi,

This is a query regarding data synchronization in elasticsearch.

We encountered a problem where we insert some data into elasticsearch at a regular interval. Now, immediately after data insertion, we run a search query for that particular data but the search query fails to find it even when the data is visible on kibana dashboard. The same query, when ran after a minute or so, is able to find the data.  
I am not sure, but maybe this is because of data synchronization takes some time?If yes, is there any parameter that could be set/unset to reduce data synchronization time?  
If no, what can we do in order to succefully be able to run search query?

Best Regards,  
Akshat

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [August 28, 2020, 4:42pm UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/2 "2020-08-28T16:42:12Z")

</div>

Check `refresh_interval` [here](https://www.elastic.co/guide/en/elasticsearch/reference/master/index-modules.html#index-refresh-interval-setting)

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [August 29, 2020, 6:37am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/3 "2020-08-29T06:37:10Z")

</div>

> "the search query fails to find it even when the data is visible on kibana dashboard"

But refresh\_interval makes no sense when Kibana shows it. But be SURE you see it in Kibana and not just some similar data, as I've made this mistake.

No way Kibana can show the data unless it's already refreshed, so makes me wonder if you have time sync issue, the timestamps and your query may not align so your query and Kibana using slightly different windows (like from 11:23-11:44) but after a few minute you see data - we see this a lot in our system with wrong timezones, though of course that's after 1 or more hours.

If you can repeat this, suggest using a direct cURL or Devtools query instead, which will have a lot fewer parameters and added 'stuff' from Kibana - just pull last docs sorted by descending timestamp to see what ES really has, compare to your query, etc.

Of course, just for testing, you can temporarily adjust referesh\_interval shorter (watch loads if you have high index rates) down to the default of 1s to see if helps, but it should not matter if Kibana or cURL really can see the data.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 29, 2020, 6:41am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/4 "2020-08-29T06:41:52Z")

</div>

What does the query look like?

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [September 3, 2020, 9:24am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/5 "2020-09-03T09:24:34Z")

</div>

@ylasri @Steve_Mushero thanks for your inputs, I'll look into what you have suggested.

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [September 3, 2020, 9:25am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/6 "2020-09-03T09:25:59Z")

</div>

@Christian_Dahlqvist Sorry for the delayed response.  
here is the query:

```auto
curl -k -u admin:kibadmin -H "Content-Type: application/json" https://elasticsearch.logging.svc.cluster.local:9200/alarms*/_search --insecure -d '
{
"query": {
"bool": {
"must": [
{
"match": {
"specificProblem":

{ "query": "1597240364" }

}
}
]
}
}
}' |jq 

```

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [September 10, 2020, 9:07am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/8 "2020-09-10T09:07:20Z")

</div>

Hi @Christian_Dahlqvist  
Any suggestions ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 10, 2020, 9:11am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/9 "2020-09-10T09:11:32Z")

</div>

What is your refresh interval for the index? How many nodes do you have in the cluster?

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [September 21, 2020, 5:30am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/10 "2020-09-21T05:30:42Z")

</div>

Hi @Christian_Dahlqvist,  
refresh\_interval is not set manually, it should take the default value for it i.e. 1 second.  
We have 3 Master, 3 client and 2 data nodes, overall usually we have 13-14 nodes in the kubernetes cluster.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 21, 2020, 5:35am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/11 "2020-09-21T05:35:33Z")

</div>

Which version of Elasticsearch are you running?

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [September 21, 2020, 6:55am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/12 "2020-09-21T06:55:40Z")

</div>

We are using both 7.0.1 and 7.8.0.

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [September 22, 2020, 10:25am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/13 "2020-09-22T10:25:03Z")

</div>

@Christian_Dahlqvist any suggestions ?

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [September 30, 2020, 8:12am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/14 "2020-09-30T08:12:31Z")

</div>

@Christian_Dahlqvist Any update on this?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 30, 2020, 8:14am UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/15 "2020-09-30T08:14:24Z")

</div>

No. I do not have any idea what is going on.

---

<div class="post-metadata">

**Author:** ![akshatsharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akshatsharma/32/84179_2.png) [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\
**Post date:** [October 5, 2020, 12:26pm UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/16 "2020-10-05T12:26:53Z")

</div>

How can we find what might have caused the delay?  
Can anyone from elastic community help with this?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 2, 2020, 12:27pm UTC](https://discuss.elastic.co/t/data-synchronization-time-in-elasticsearch/246748/17 "2020-11-02T12:27:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.