# Strange behavior after losing a solo master node **URL:** \ **Category:** Elasticsearch\ **Created:** [November 25, 2014, 9:41am UTC](https://discuss.elastic.co/t/strange-behavior-after-losing-a-solo-master-node/20931 "2014-11-25T09:41:34Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![fred](https://avatars.discourse-cdn.com/v4/letter/f/db5fbb/32.png) [@fred](https://discuss.elastic.co/u/fred)\ **Post date:** [November 25, 2014, 9:41am UTC](https://discuss.elastic.co/t/strange-behavior-after-losing-a-solo-master-node/20931/1 "2014-11-25T09:41:34Z")
Hello, We just had something very weird on our ES cluster, and I wanted to know if this behavior was either documented or expected or... ES version: 1.0.1 1 node: master: yes http: yes data: no 3 nodes: master: no http: yes data: yes The virtual machine with the master node was destroyed, so we had to replace it with a brand new, bare one. When the new master joined the cluster, the cluster went red, all shards were marked as unassigned and started to initialize. Also all application queries we sent to the cluster failed with: [2014-11-12 21:32:00,120][DEBUG][action.search.type] [some host] [cluster][23], node[4ZYs5T2QSVG0pSRiE7zJDg], [R], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@32b3ab46] org.elasticsearch.transport.RemoteTransportException: [some host][inet[/1.2.3.4:9300]][search/phase/query+fetch] Caused by: org.elasticsearch.ElasticsearchException: java.lang.NumberFormatException: Invalid shift value (64) in prefixCoded bytes (is encoded value really an INT?) at org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:75) at org.elasticsearch.search.aggregations.support.FieldDataSource$MetaData.load(FieldDataSource.java:88) at org.elasticsearch.search.aggregations.support.AggregationContext.bytesField(AggregationContext.java:175) at org.elasticsearch.search.aggregations.support.AggregationContext.valuesSource(AggregationContext.java:126) at org.elasticsearch.search.aggregations.support.ValueSourceAggregatorFactory.create(ValueSourceAggregatorFactory.java:57) at org.elasticsearch.search.aggregations.AggregatorFactories.createSubAggregators(AggregatorFactories.java:55) at org.elasticsearch.search.aggregations.Aggregator.(Aggregator.java:78) at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.(BucketsAggregator.java:46) at org.elasticsearch.search.aggregations.bucket.SingleBucketAggregator.(SingleBucketAggregator.java:32) at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregator.(FilterAggregator.java:50) at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregator$Factory.create(FilterAggregator.java:91) at org.elasticsearch.search.aggregations.AggregatorFactories.createTopLevelAggregators(AggregatorFactories.java:127) at org.elasticsearch.search.aggregations.AggregationPhase.preProcess(AggregationPhase.java:79) at org.elasticsearch.search.query.QueryPhase.preProcess(QueryPhase.java:91) at org.elasticsearch.search.SearchService.createContext(SearchService.java:511) at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:472) at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:316) at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryFetchTransportHandler.messageReceived(SearchServiceTransportAction.java:686) at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryFetchTransportHandler.messageReceived(SearchServiceTransportAction.java:675) at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:270) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) [2014-11-12 21:32:00,121][DEBUG][action.search.type] [some host] All shards failed for phase: [query\_fetch] Unfortunately, I was unable to reproduce it on another cluster. So here are my guesses: - There might have been some transient information I've lost with the master node - Never have less than 2 master nodes, even temporarily. - We have incidentally ran into mapping type gotchas as explained here. [http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/mapping.html](http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/mapping.html) I'm still bothered by the reshard and lose your mapping thing as I could not reproduce it. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2dc96928-04bf-4b18-b599-b33e6609ae1c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2dc96928-04bf-4b18-b599-b33e6609ae1c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:47am UTC](https://discuss.elastic.co/t/strange-behavior-after-losing-a-solo-master-node/20931/2 "2017-07-06T00:47:48Z")
Closed on July 6, 2017, 12:47am UTC