Stuck with "re-syncing mappings with cluster state for types ..." after upgrade 0.90.9 to 1.4.2

Elastic Stack Elasticsearch
1

Hi List,

we have recently upgraded our ES cluster (12 nodes, ~900GB data) from
0.90.9 to 1.4.2. We did a restart-upgrade, so backup data and then start
the new ES version with the old data directories.

Since then we're seeing lot of messages like "re-syncing mappings with
cluster state for types ..." in the log of the master node, about 5-10
every second (Once for each index). The load and network traffic on the
machines also seems to have increased quite a bit.

Some googling showed me various older (0.17/0.18) questions about this, and
that it's related to serialization of the mapping. Our mapping is here:

One suggested quick fix i found was to close/re-open the affected indices.
Other suggestions? Or tips how to debug why this is going wrong, and what
to change in the mapping?

How serious is this? Does this need immediate attention, or should the
cluster survive for a while?

best regards,
Eike Dehling

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/de51ebc3-ca47-475c-914d-91d7a747e69e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Did you upgrade Java as well as ES? What version are you on?

On 20 January 2015 at 04:55, Eike Dehling eike@buzzcapture.com wrote:

Hi List,

we have recently upgraded our ES cluster (12 nodes, ~900GB data) from
0.90.9 to 1.4.2. We did a restart-upgrade, so backup data and then start
the new ES version with the old data directories.

Since then we're seeing lot of messages like "re-syncing mappings with
cluster state for types ..." in the log of the master node, about 5-10
every second (Once for each index). The load and network traffic on the
machines also seems to have increased quite a bit.

Some googling showed me various older (0.17/0.18) questions about this,
and that it's related to serialization of the mapping. Our mapping is here:
elasticsearch 1.4.2 mapping · GitHub

One suggested quick fix i found was to close/re-open the affected indices.
Other suggestions? Or tips how to debug why this is going wrong, and what
to change in the mapping?

How serious is this? Does this need immediate attention, or should the
cluster survive for a while?

best regards,
Eike Dehling

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/de51ebc3-ca47-475c-914d-91d7a747e69e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/de51ebc3-ca47-475c-914d-91d7a747e69e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9zVCd1NPbMQqpoMeEW5oyfvonP-syf7h8kATA1cAceoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks for the response. We did not upgrade java, we're running java
1.7.0_45, and Elasticsearch 1.4.2.

For anyone else running into this: We did manage to "solve" the issue by
closing and re-opening all indices. After that the continuous
re-synchronization stopped. Certain events start it again though: For
example moving a shard. Closing/re-opening an index then "solves" it again
though.

regards, Eike

On Mon, Jan 19, 2015 at 10:32 PM, Mark Walkom markwalkom@gmail.com wrote:

Did you upgrade Java as well as ES? What version are you on?

On 20 January 2015 at 04:55, Eike Dehling eike@buzzcapture.com wrote:

Hi List,

we have recently upgraded our ES cluster (12 nodes, ~900GB data) from
0.90.9 to 1.4.2. We did a restart-upgrade, so backup data and then start
the new ES version with the old data directories.

Since then we're seeing lot of messages like "re-syncing mappings with
cluster state for types ..." in the log of the master node, about 5-10
every second (Once for each index). The load and network traffic on the
machines also seems to have increased quite a bit.

Some googling showed me various older (0.17/0.18) questions about this,
and that it's related to serialization of the mapping. Our mapping is here:
elasticsearch 1.4.2 mapping · GitHub

One suggested quick fix i found was to close/re-open the affected
indices. Other suggestions? Or tips how to debug why this is going wrong,
and what to change in the mapping?

How serious is this? Does this need immediate attention, or should the
cluster survive for a while?

best regards,
Eike Dehling

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/de51ebc3-ca47-475c-914d-91d7a747e69e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/de51ebc3-ca47-475c-914d-91d7a747e69e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/4Eq6no-j8v0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9zVCd1NPbMQqpoMeEW5oyfvonP-syf7h8kATA1cAceoQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9zVCd1NPbMQqpoMeEW5oyfvonP-syf7h8kATA1cAceoQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Met vriendelijke groet,
Kind regards,

Eike Dehling
Lead Developer

Buzzcapture
Herengracht 180, 1016 BR, Amsterdam

T: +31 (0)20 3200377
M: +31 (0)6 45144840

LinkedIn | @buzzcapture

Recent

06-01-2015: Nationale Nederlanden, OLVG/Sint Lucas Andreas en H&M starten
met geïntegreerde monitoring
04-01-2015: Vacatures http://www.buzzcapture.com/tag/vacature/:
Buzzcapture breidt uit en zoekt een Media analist, Developer en Stagiair
01-01-2015: Buzzcapture volop in media
http://www.buzzcapture.com/2015/01/kan-social-media-ook-dit-jaar-afvaller-wie-de-mol-voorspellen/
dankzij
nieuwe Wie is de Mol?-analyse
01-01-2015: Download
http://www.buzzcapture.com/2014/12/de-buzzcapture-contentkalender-2015/ nu
de Buzzcapture Contentkalender 2015
23-12-2014: Veel bestaande klanten integreren print
https://www.buzzcapture.com/blog/buzzcapture-biedt-en-online-geintegreerde-monitoring-aan/
content
in Social Media Dashboard

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMp7VWQ5sF%2BDS%3DQkA7GAQqPXC45sZjAJzTQbshJsZmYXjjv7FQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.