What is causing refreshes?

Elastic Stack Elasticsearch
1

I am using elasticsearch 5.6.4 and I have a very update heavy workload. I have my refresh_interval set to 10s. However, I'm seeing hundreds of concurrent refreshes going on (with a max refresh time of ~5s). Is there a way to determine what is triggering these refreshes?

Based on this, I've tried setting up a lock system so that the same document is only updated once every five minutes, but I'm still seeing this large amount of refreshes.

I am making all the updates as scripted upserts through the bulk api with refresh=false.

2

The updates are causing the refreshes. See this and this and this.

3

Yes. The updates are definitely the cause, so I suppose my lock system might not be working as intended. I'm wondering if there's any way to see which doc updates are causing an index refresh

4

if you want to know which index is getting the updates, you can look in the index stats to see which one has high merge activity.

if you want to know which documents are getting the updates, you could turn on slow query logging for a little while with a really low threshold.

1 Like
5

Oh! That's a really good idea with the slow log. I'll give that a shot. Thanks!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.