Global ordinals on high cardinality fields with large indices


I've got a daily index with a couple of high cardinality fields. Even after reading some threads and the documentation available, I'm not sure how should I proceed. The index has ~600M docs in it every day and the most requested HC field has ~20M unique values. The index has 6 shards on 8 nodes. I'm trying to lower the general memory usage and I'm not sure if adding eager global ordinals to the affected fields' mapping would help in any way or if I change the execution hint to map would do any harm. Unfortunately, the queries are aggregating those unique values to different term buckets almost every time.
Any suggestions?


If daily indices are no longer written to you can reduce memory usage by forcemerging them down to a single segment as described in this webinar. It can take a while and result in a lot of disk I/O though.

1 Like

Thank you @Christian_Dahlqvist !
Even if it'd be a segment that's 300GB in size?

Is that your shard size??

Nope, I have 6 shards so one shard would only be around 50-60GB.

Then will be a segment per shard, but shards that size will take some time to forcemerge.

I see, thanks! I'll watch the webinar soon too. Would eager global ordinals help for the actively written shards?

I do not think so.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.