Upgraded from 7.1.0 to 7.3.1 and stoped working

Elastic Stack Elasticsearch
1

After upgrading ES to 7.3.1 when I start ES I've got in log:
What can be wrong?

[2019-09-09T16:27:45,286][DEBUG][o.e.a.s.TransportSearchAction] [HV8] All shards failed for phase: [query]
[2019-09-09T16:27:45,286][DEBUG][o.e.a.s.TransportSearchAction] [HV8] All shards failed for phase: [query]
[2019-09-09T16:27:45,286][WARN ][r.suppressed             ] [HV8] path: /.kibana_task_manager/_search, params: {ignore_unavailable=true, index=.kibana_task_manager}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:305) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:139) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:264) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:105) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$1(InitialSearchPhase.java:251) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase$1.doRun(InitialSearchPhase.java:172) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_181]
	at java.lang.Thread.run(Unknown Source) [?:1.8.0_181]
[2019-09-09T16:27:45,286][WARN ][r.suppressed             ] [HV8] path: /.kibana_task_manager/_search, params: {ignore_unavailable=true, index=.kibana_task_manager}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:305) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:139) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:264) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:105) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$1(InitialSearchPhase.java:251) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase$1.doRun(InitialSearchPhase.java:172) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_181]
	at java.lang.Thread.run(Unknown Source) [?:1.8.0_181]
[2019-09-09T16:27:58,143][DEBUG][o.e.a.s.TransportSearchAction] [HV8] All shards failed for phase: [query]
[2019-09-09T16:27:58,143][WARN ][r.suppressed             ] [HV8] path: /.reporting-*/_search, params: {index=.reporting-*}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:305) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:139) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:264) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:105) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$1(InitialSearchPhase.java:251) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase$1.doRun(InitialSearchPhase.java:172) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_181]
	at java.lang.Thread.run(Unknown Source) [?:1.8.0_181]
[2019-09-09T16:28:06,782][WARN ][o.e.m.j.JvmGcMonitorService] [HV8] [gc][62] overhead, spent [3.4s] collecting in the last [3.6s]
[2019-09-09T16:28:11,299][WARN ][o.e.m.j.JvmGcMonitorService] [HV8] [gc][64] overhead, spent [2.9s] collecting in the last [3.5s]
[2019-09-09T16:28:17,175][WARN ][o.e.m.j.JvmGcMonitorService] [HV8] [gc][67] overhead, spent [3.5s] collecting in the last [3.7s]
2

What is in your kibana logs and can you include your es and kibana configuration?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.