Stale index

Elastic Stack Elasticsearch
1

Hi Guys,

Is there some way to know if an index is stale?

Regards,
Anton

2

What do you mean by "stale index"?

3

Sorry for being unclear, I mean the time when a index is not up to date
with the latest writes. Would be great to be able to get some kind of
notification or being able to check if a index is up to date.

Like in RavenDb, you can check the status of an index to see if it's up to
date with all the latest documents or you can get a notification when
documents of a certain type has been indexed. With this type of
functionality we can inform the end user when a large import of data is
available for searching.

Thanks,
Anton

4

Data that you have fed to ES and received an acknowledgement for will be available for queries after the next refresh of the index, which is initiated explicitly via an API call or periodically every few seconds (configurable per index).

So if you know the refresh interval you have an upper bound for how stale queries can be, and if you want an immediate refresh that's also an option (but doing it too often hurts performance).

To improve performance it's a common practice to disable periodic refreshes for the duration of a bulk indexing operation.

5

Thank you for your response! Wow, I didn't realize that data is available
directly after the refresh interval, even so if the index is really large?

Great tip to disable periodic refreshes for the duration of a bulk indexing
operation. I guess that is something which can be configured per index
through the API?

Thanks,
Anton

6

You can update the refresh.interval via the index APIs, yep.