Forcemerge concerns

Elastic Stack Elasticsearch
1

Hi!
I need to reduce de size of a database by shrinking one field from a document. The problem is: the size of this database is huge. Also, I need to not reduce the performance of it significantly. I'm wondering if I can reindex the document and use a _forcemerge. BUT I saw that I can't write while doing that, so it's not a good idea. Can someone guide me to the best practice about removing versions from a document after reindex in database yet being used
to write?

Thanks!!

2

Where did you read that you cannot write while a merge is happening? Elasticsearch is doing merging in the background all the time while indexing happens.

3

Hi there! Thanks for the answer. I read it here:
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/indices-forcemerge.html

Screenshot_20191216-071640
Screenshot_20191216-0716401080×1920 267 KB

Anyway, do you know what is the best practice for this kind of task?
Thank you!

4

the call will block until the merge is complete

This is only about the HTTP call send to Elasticsearch, not about any write operations.

If you need to remove a certain field, you should take a look at reindex or update-by-query

5

Actually I am more concerned about this:

"Force merge should only be called against read-only indices . Running force merge against a read-write index can cause very large segments to be produced (>5Gb per segment), and the merge policy will never consider it for merging again until it mostly consists of deleted docs. This can cause very large segments to remain in the shards."

6

yes, this is an important advice. Only run force-merge if the index will not see any updates. Otherwise merges will be running at some point in the background anyway.

To me, reindex (without a forcemerge) sounds like a good idea to start with.

7

Thanks for the solution!
I havent mentioned that, but my db has like 7tb of data.
Do you still consider it a good idea?

8

as usual there is no clear answer, you might need more space during reindex until save spacings kick in (especially when indexing into a new index), so you need to be sure you will be able to sustain that.

9

Thank you a lot!

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.