Hi Team,
I was trying to delete the old indices and ran into an issue.
Now i am getting the below message in elastic search logs?
Could you please help me out here?
[2019-06-26T14:07:21,613][INFO ][o.e.n.Node ] [vAswxKt] starting ...
[2019-06-26T14:07:21,898][INFO ][o.e.t.TransportService ] [vAswxKt] publish_address {10.188.210.34:9300}, bound_addresses {10.188.210.34:9300}
[2019-06-26T14:07:22,344][WARN ][o.e.b.BootstrapChecks ] [vAswxKt] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2019-06-26T14:07:22,638][INFO ][o.e.h.n.Netty4HttpServerTransport] [vAswxKt] publish_address {10.188.210.34:9200}, bound_addresses {10.188.210.34:9200}
[2019-06-26T14:07:22,639][INFO ][o.e.n.Node ] [vAswxKt] started
[2019-06-26T14:07:29,850][INFO ][o.e.g.GatewayService ] [vAswxKt] recovered [146] indices into cluster_state
[2019-06-26T14:08:31,203][DEBUG][o.e.a.s.TransportSearchAction] [vAswxKt] All shards failed for phase: [query]
[2019-06-26T14:08:31,204][WARN ][r.suppressed ] path: /dmarc_aggregate*/_search, params: {index=dmarc_aggregate*}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:274) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:132) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:243) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:107) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$4(InitialSearchPhase.java:205) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase$1.doRun(InitialSearchPhase.java:184) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.2.jar:6.2.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_192]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_192]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_192]
[2019-06-26T14:08:32,227][DEBUG][o.e.a.s.TransportSearchAction] [vAswxKt] All shards failed for phase: [query]
[2019-06-26T14:08:32,227][WARN ][r.suppressed ] path: /dmarc_aggregate*/_search, params: {index=dmarc_aggregate*}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:274) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:132) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:243) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:107) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$4(InitialSearchPhase.java:205) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase$1.doRun(InitialSearchPhase.java:184) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.2.jar:6.2.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_192]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_192]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_192]
[2019-06-26T14:08:35,244][DEBUG][o.e.a.s.TransportSearchAction] [vAswxKt] All shards failed for phase: [query]
[2019-06-26T14:08:35,244][WARN ][r.suppressed ] path: /dmarc_aggregate*/_search, params: {index=dmarc_aggregate*}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:274) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:132) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:243) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:107) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$4(InitialSearchPhase.java:205) ~[elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.action.search.InitialSearchPhase$1.doRun(InitialSearchPhase.java:184) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.2.jar:6.2.2]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.2.2.jar:6.2.2]
at
GET /_cat/indices gives the below output
red open winlogbeat-6.6.1-2019.05.11 9Qejm9hXSGaZXuuDfOJstg 1 1
red open paloalto-web-logs xG0ah1N5QXaVAmIfHoz07g 5 1
red open nifi-stats uJeSz4z6SiWtgD-Yz_r4jA 1 1
red open winlogbeat-6.6.1-2019.03.20 IdSGFA23Q9CIyCjt0uHMQw 1 1
red open winlogbeat-6.6.1-2019.05.09 oMolshskToaTK7sassDNwg 1 1
red open winlogbeat-6.6.1-2019.05.20 Ac36sHcxQxGbgddopghiuA 1 1
red open winlogbeat-6.6.1-2019.02.27 pUqXaRO-SSiaLoskkVcykw 1 1
red open dmarc_aggregate_dmarc-2019-06-13 dXGfUOpSRauwaRkalZJ1Gw 1 1
red open dmarc_aggregate_dmarc-2019-06-15 pD7IfdluRgmDPFbRqD7m9A 1 1
red open winlogbeat-6.6.1-2019.06.25 _HfkDiKEQhiaESp-LLf7AQ 1 1
red open winlogbeat-6.6.1-2019.06.04 suJVgcrbTSmwqZyjstKJng 1 1
red open dmarc_aggregate_dmarc-2019-06-18 Jhi1TSQASkOhSfi_f6gZvA 1 1
red open winlogbeat-6.6.1-2019.05.05 3dOdoe-WSSK-m0XdekZneg 1 1
red open winlogbeat-6.6.1-2019.04.21 2jEQynufSUmV8D9AJwAhQA 1 1
red open winlogbeat-6.6.1-2019.04.23 dJ6XBVwRQ9qrWhpHpAMBgw 1 1
red open winlogbeat-6.6.1-2019.05.03 W5Tdf-c8RS-vWiEfshrdig 1 1
red open winlogbeat-6.6.1-2019.06.19 UvAO_HeMRgiAWqffsaNo8A 1 1
red open dmarc_aggregate_dmarc-2019-06-23 f5dq16YvStWb5ibPwnMXew 1 1
red open winlogbeat-6.6.1-2019.06.17 yg-US-cuTlm20RuAER4Y9Q 1 1
red open dmarc_aggregate_dmarc-2019-06-14 mzZadbM6SMivFNtIo9jqPA 1 1
red open filebeat-6.2.2-2019.02.14 YWHj0agDRvOC8Dj3Agm9hg 3 1
red open winlogbeat-6.6.1-2019.06.21 5HsumnRRRL-pIxg_mms67g 1 1
red open winlogbeat-6.6.1-2019.03.24 qhpFpxAeR5-OaWVy1oNIMA 1 1
red open winlogbeat-6.6.1-2019.05.18 bLgtM1_1RAG0ayaRXybyxA 1 1
red open filebeat-6.2.2-2019.02.19 nUb3C3LYSjWW59AXQ6Gqfw 3 1
red open winlogbeat-6.6.1-2019.04.16 iBKjN-e9QUq4CPpFvbJY5Q 1 1
red open winlogbeat-6.6.1-2019.06.15 SpunZ4GeT-ydYMhtjka69w 1 1
red open winlogbeat-6.6.1-2019.04.25 hLsp0sTCQv2mDliWlv0mXQ 1 1
red open filebeat-6.2.2-2019.02.28 6o6eH4YDRNClrDc5w4rgyg 3 1
yellow open test dE8-jxZRR9SN9rpsZuuyvQ 5 1 1 0 4.5kb 4.5kb
red open filebeat-6.2.2-2019.02.24 ROpOR5onT7GRw48HybtDsQ 3 1
red open winlogbeat-6.6.1-2019.06.23 2Kdd1G91T6q2mY-dyItygA 1 1
red open winlogbeat-6.6.1-2019.04.30 OtFUx2rQS-CSNhBevIkbVg 1 1
red open filebeat-6.2.2-2019.02.21 aknCZbUTRSGZGTwEsE3Hkg 3 1
red open winlogbeat-6.6.1-2019.04.02 GoiMPPBuS7KK4Ii1yv3w8w 1 1
red open winlogbeat-6.6.1-2019.06.05 tKaoo7rNT_WNyZkhx6kAPQ 1 1
red open filebeat-6.2.2-2019.03.04 kBowFA1uRISvicoZA1X-Gw 3 1
red open winlogbeat-6.6.1-2019.05.23 dCUJg_UmQpu6dJoBDi_ibw 1 1
red open winlogbeat-6.6.1-2019.05.22 HuS1wpFYRcyaytGm5wbm0Q 1 1
red open winlogbeat-6.6.1-2019.06.16 LO07cNWcSc-VFRsY3G2brA 1 1
red open nifi-queue-stats AjY8Qt8sRiSDVjs3DcrzhQ 2 1
green open json ABWhJBF-S2CplUBP_dWXSw 5 0 0 0 1.2kb 1.2kb
red open winlogbeat-6.6.1-2019.06.06 sPLxwRDhR3KDi6jKYbnVJg 1 1
red open winlogbeat-6.6.1-2019.04.15 uI7jqLH_QFa4w32CHT9Jzw 1 1
red open winlogbeat-6.6.1-2019.05.31 cyL-MzCiREaIgQsrpAG55A 1 1
red open filebeat-6.2.2-2019.02.22 DU5EFtpITHOEQiBuspVkpA 3 1
red open winlogbeat-6.6.1-2019.06.08 tQr2ahPHTxWCL29mnuARVA 1 1
red open winlogbeat-6.6.1-2019.05.15 c8ixDbD9RfG6pBdktNCSBQ 1 1
red open winlogbeat-6.6.1-2019.03.26 mH9AVy0qQiWaDyiwqyXjUA 1 1
red open winlogbeat-6.6.1-2019.05.13 E527uSu-RSaZ7Kr1U-tCMQ 1 1
red open winlogbeat-6.6.1-2019.06.07 oUY75v-ARB6wPBeacHtOSw 1 1
red open winlogbeat-6.6.1-2019.06.22 aD7VHeZJTLCec2iRfIZP4A 1 1
red open winlogbeat-6.6.1-2019.06.09 wqLzV_uuTLiYANzg7tsOCA 1 1
red open winlogbeat-6.6.1-2019.05.25 AlHOVIcNTv64hwQ9mvXDZA 1 1
red open winlogbeat-6.6.1-2019.03.22 D5teR5KbRKq6I0W4Ez6pXQ 1 1
red open filebeat-6.2.2-2019.02.25 SmuTHwhiQkyyuwEMvILuVA 3 1
red open filebeat-6.2.2-2019.02.16 JP3Lz3swRheET2smuC3jbQ 3 1
red open winlogbeat-6.6.1-2019.03.28 Kp4ib0SNSPmdfmhnuZ9ouA 1 1
red open filebeat-6.2.2-2019.03.03 pP9V-60_S4aV716nV2ETbw 3 1
red open winlogbeat-6.6.1-2019.05.19 b_xycYagROavxmbdG0s2og 1 1
red open winlogbeat-6.6.1-2019.05.16 siJARfeSRZWF037kLBaUcA 1 1
red open winlogbeat-6.6.1-2019.05.01 jHLcRq5CRcafRkbCjAVmMg 1 1
red open dmarc_aggregate_dmarc-2019-06-12 1_9YTVpyRYCAEHzJ-vM9HQ 1 1
red open filebeat-6.2.2-2019.02.11 aGjQto07RJK_NNA1ls_49A 3 1
red open filebeat-6.2.2-2019.02.27 vSTd-e9uS_KIqmkOqzZitw 3 1
red open winlogbeat-6.6.1-2019.04.19 TrvMBPtKSPKpFVGBDF_OVA 1 1
red open winlogbeat-6.6.1-2019.03.16 iFxyfZiBSpqVfPKn4VLkDw 1 1
red open winlogbeat-6.6.1-2019.03.15 -6WfRalbQBCk-HY-iARjlA 1 1
Hey,
searching against the dmarc_aggregate indices failed. Can you share a full stack trace from the client side? See https://www.elastic.co/guide/en/elasticsearch/reference/7.2/common-options.html#common-options-error-options
Thanks!
--Alex
Hi Alexander,
Thanks a lot for your response.
I was very impatient so i removed elasticserch and reconfigured it. That solved the problem.
Thanks
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.