Avoid index ballooning by updates and deletes

Elastic Stack Elasticsearch
1

Hi,
I'm struggling with having an index that has some fields that are frequently updated and documents that are deleted. I want to avoid ballooning the index for obvious reasons. I tried to go with freezing the index and reindex it, but it seems you can't reindex a frozen (read-only) index, which is weird.
So I need a solution where I can dispose of the unused documents while it's continuously available for queries. Thankfully, the index isn't huge it's less than 1GB.

Thank you!
YvorL

2

For an index that small I would imagine that auto merging should keep things in check. Is it not?

3

I don't have that much experience with this type of index. I usually go with daily/monthly ones. I remembered that you can't really delete from an index and updates will create new versions. Also, that you can't force merge active indices. I may have overthought this but I have to be sure that this won't be an issue in production.
@warkolm could you point me to documentation or a blog post where the auto merging is explained so I know what to expect? I mostly found force merge docs and git issues with auto merging when I googled it.
Thank you!

4

Let me counter that by asking - what do you mean by ballooning here? What are you seeing that has you worried?

5

My inexperience :slight_smile: As I mentioned, I don't have any experience with continuously updated indices and I always think of the worst. For example, if every doc in the index is updated X times (4-10) in a day it'll double its size very quickly and performance drops.

6

I think you're worrying about nothing worth worrying about.

An updated document will delete the original, but merging will get rid of it. You won't have massive size increases.

1 Like
7

Thank you!

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.