I have an out of memory problem in ES 0.17.5.
My cluster contains two data nodes and a lot of short living "test nodes",
that doesn't store any data only connect to cluster to do some operations on
Every "test node" creates its own index with some types and complex mapping
and cleans this index before shutdown.
After a while my cluster is broken - one of the data nodes throws OOM. It
contains then ~100 empty indices.
I analyzed heap dump of broken node and almost whole memory is used by
variable "workQueue" which is of type LinkedBlockingQueue and contains 1798
items of type org.elasticsearch.cluster.service.InternalClusterService$2,
with size ~640kB each
where updateTask.newState.metaData.indices has size ~491kB and
updateTask.newState.routingTable.indicesRouting has size ~102kB.
If this update task contains information about all indices and its mapping
then its size is ok, but why there are so many update tasks in this queue?