Does the cluster state size impact performance?

In my cluster running 0.20.1, the message returned from /_cluster/state is
6.8M and growing. Will that cause a problem?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey,

In short, this not a problem but rather a necessity to keep your cluster
intact with all its information.

the cluster state contains lots of metadata about all your indices, shards,
their mappings as the current cluster status. This means, by adding more
indices the complete cluster state data also grows as more metadata is
added. The cluster state is also being replicated from the master to all
other nodes on changes.

In case you are not adding indices or not adding documents with new fields
(which results in new mapping entries) or you do not add new nodes with
more shards you might want to elaborate what is causing this though.

Hope it helps..

--Alex

On Wed, Mar 27, 2013 at 10:50 PM, Telvis Calhoun Jr. telvis07@gmail.comwrote:

In my cluster running 0.20.1, the message returned from /_cluster/state is
6.8M and growing. Will that cause a problem?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I am adding indices and each index has 30+ type mappings and each
mapping has 10+ fields. During training, one of the trainers (Yuri I
think) mentioned that adding mappings will grow the state and that it
could have a performance impact if there are too many.

On Thu, Mar 28, 2013 at 4:14 AM, Alexander Reelsen alr@spinscale.de wrote:

Hey,

In short, this not a problem but rather a necessity to keep your cluster
intact with all its information.

the cluster state contains lots of metadata about all your indices, shards,
their mappings as the current cluster status. This means, by adding more
indices the complete cluster state data also grows as more metadata is
added. The cluster state is also being replicated from the master to all
other nodes on changes.

In case you are not adding indices or not adding documents with new fields
(which results in new mapping entries) or you do not add new nodes with more
shards you might want to elaborate what is causing this though.

Hope it helps..

--Alex

On Wed, Mar 27, 2013 at 10:50 PM, Telvis Calhoun Jr. telvis07@gmail.com
wrote:

In my cluster running 0.20.1, the message returned from /_cluster/state is
6.8M and growing. Will that cause a problem?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/UNeh8EnZaq4/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On Thu, 2013-03-28 at 11:34 -0400, Telvis Calhoun Jr. wrote:

I am adding indices and each index has 30+ type mappings and each
mapping has 10+ fields. During training, one of the trainers (Yuri I
think) mentioned that adding mappings will grow the state and that it
could have a performance impact if there are too many.

Think that:

  • the cluster state is on every node
  • has to be transferred to every node every time it changes

So yes, very big cluster states can have a performance impact.

We've seen states with hundreds of thousands of aliases, which can take
a couple of seconds to update

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.