Why Elasticsearch Mapping Explosion can causes OOM？

Elastic Stack Elasticsearch
1

I think index Mapping is a large JSON object. Even if each node sends a copy, the memory usage is limited. Why does it cause OOM?

2

It also depends on the total number of shards you have on your cluster.

What is the output of:

GET /
GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v

If some outputs are too big, please share them on gist.github.com and link them here.

1 Like
3

Each field is much more than just the few bytes that describe it in the JSON representation of the mapping - Elasticsearch also has to maintain a mechanism to index into each field and then to retrieve information from the field for searches, and these things all consume additional memory. It's a bit more memory-efficient in recent versions (7.16+) so if you're experiencing OOMs due to too many fields you could try upgrading. But still, an excess of fields makes indexing and searching less efficient so it's best to keep your field count under control even if you don't hit OOMs.

1 Like
4

Whether there is a range for how much memory a mapping field can consume. This allows the user to estimate the memory usage of the mapping.

5

The resource usage also depends on the data held in the index. I don't think it's something you can compute on a field-by-field basis.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.