Elasticsearch[7.7] Does not update data or takes time to update the data

Elastic Stack Elasticsearch
1

In my application, we are updating more than one document at a time for a index_x. When queried for updated documents, it does not return updated data for few document, immediately. e.g. If I am updating 10 documents, 1 document update is not retrieved immediately but after 10 to 15 mins. It has been observed that, the missed data is updated after 10 or 15 mins. For index_x, we have set refresh parameter as default i.e. 1s.
I am using Elasticsearch JAVA repository to update the document. I am updating whole document instead of selected fields. I observed that though I have sent the query to update document, it has not updated till next day also. I checked in Kibana as well. It returns old data. Still the data shows old.

Below are the app configurations for app:

JAVA: 11
Spring Boot: 2.7.11
Spring Data Elasticsearch: 4.2.12
Elasticsearch client - 7.12.1
Elasticsearch version on AWS - 7.7

2

Welcome!

That's the expected behavior if you are not refreshing the index (refresh API).

3

Thank you for the quick response. Though I am not refreshing the index manually, it should get refreshed in 1s as per the refresh_interval(set to 1s). What about the document which were not at all get updated even after 2 days? I tried to run refresh API for my index but it seems the update it-self is not executed as I am not getting updated data after refresh API or even after 2 days.

4

If you ran a refresh on the index you are searching on and the document has not been updated, this is most likely because the update did not work for that document. You should see a reject somewhere.

BTW you are using a version which is too old. Please update to 7.17.18.

5

Where I can see this rejection? At least I couldn't see that in app log. Do I need to check in Elasticsearch log? or in Kibana? Also if you could let me know the sample response for reject update that would be helpful.
And yes we planning for upgrade in this year.

6

Somewhere in your code...
How are you running the update?

7

I am using ElasticsearchRepository.saveAll(document).

8

Are you checking the result? I don't see your code here. So it would help if you share more.
Anyway, ElasticsearchRepository is not a class made by Elastic but from Spring Data project.

I don't know if/how you can add more logs but this is something you probably need to check.

Some advices though:

  • Upgrade Elasticsearch client to 7.17.18
  • Check if upgrades are available for Spring data elasticsearch
  • If possible move to the new Elasticsearch Java API Client.

What I can tell:

If the response does not show any rejection, if no exception is thrown, and if the code actually send a new version of the document, the document will be immediately visible if you run a GET by id (whatever the refresh is).

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.