# Need help in v7.9.2 cluster configuration of 3 nodes **URL:** \ **Category:** Elasticsearch\ **Created:** [October 14, 2020, 8:15pm UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101 "2020-10-14T20:15:29Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![sges](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@sges](https://discuss.elastic.co/u/sges)\ **Post date:** [October 14, 2020, 8:15pm UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/1 "2020-10-14T20:15:29Z")
Hello, I'm setting up a cluster with elasticsearch v7.9.2 and for that I'm setting up a 3 node elasticsearch cluster environment on separate VM's. On performing search I'm seeing IllegalStateException in the logs. I need some help in checking what changes would be required in the below elasticsearch.yml configuration of respective nodes, _Node 1:_ ```auto cluster.name: es792 node.name: 10.123.234.1 node.master: true node.data: true network.host: 10.123.234.1 http.port: 9200 transport.bind_host: 10.123.234.1 transport.tcp.port: 9300 discovery.seed_hosts: ["10.123.234.1","10.123.234.2","10.123.234.3"] cluster.initial_master_nodes: [10.123.234.1] xpack.ml.enabled: false xpack.security.enabled: false ``` _Node 2:_ ```auto cluster.name: es792 node.name: 10.123.234.3 node.master: true node.data: true network.host: 10.123.234.3 http.port: 9200 transport.bind_host: 10.123.234.3 transport.tcp.port: 9300 discovery.seed_hosts: ["10.123.234.1","10.123.234.2","10.123.234.3"] xpack.ml.enabled: false xpack.security.enabled: false ``` _Node 3:_ ```auto cluster.name: es792 node.name: 10.123.234.2 node.master: true node.data: true network.host: 10.123.234.2 http.port: 9200 transport.bind_host: 10.123.234.2 transport.tcp.port: 9300 discovery.seed_hosts: ["10.123.234.1","10.123.234.2","10.123.234.3"] xpack.ml.enabled: false xpack.security.enabled: false ``` **Observing the below exceptions,** ``` 'org.elasticsearch.transport.RemoteTransportException: [10.123.234.1][10.123.234.1:9300][internal:cluster/coordination/join] Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: incoming term 1 does not match current term 2 at org.elasticsearch.cluster.coordination.CoordinationState.handleJoin(CoordinationState.java:225) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.cluster.coordination.Coordinator.handleJoin(Coordinator.java:1013) ~[elasticsearch-7.9.2.jar:7.9.2] at java.util.Optional.ifPresent(Optional.java:183) ~[?:?] Exception caught on transport layer [Netty4TcpChannel{localAddress=/10.123.234.2:9300, remoteAddress=/10.123.234.1:61631}], closing connection java.lang.IllegalStateException: transport not ready yet to handle incoming requests at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHa [2020-10-15T01:06:03,576][INFO][o.e.n.Node] [10.123.234.1] started [2020-10-15T01:07:27,857][WARN][o.e.t.TcpTransport] [10.123.234.1] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.123.234.1:63544, remoteAddress=10.123.234.3/10.123.234.3:9300}], closing connection java.lang.IllegalStateException: Message not fully read (response) for requestId [225], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.transport.TransportService$6@2d61b747], error [false]; resetting at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:124) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.2.jar:7.9.2] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] 2020-10-15T01:07:27,857][WARN][o.e.t.TcpTransport] [10.123.234.1] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.123.234.1:63548, remoteAddress=10.123.234.2/10.123.234.2:9300}], closing connection java.lang.IllegalStateException: Message not fully read (response) for requestId [229], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.transport.TransportService$6@7646f4d6], error [false]; resetting at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:124) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.2.jar:7.9.2] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] [2020-10-15T01:07:27,873][WARN][o.e.t.TcpTransport] [10.123.234.1] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.123.234.1:63546, remoteAddress=10.123.234.2/10.123.234.2:9300}], closing connection java.lang.IllegalStateException: Message not fully read (response) for requestId [226], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.transport.TransportService$6@6d5264b6], error [false]; resetting at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:124) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.2.jar:7.9.2] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 15, 2020, 8:46am UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/2 "2020-10-15T08:46:24Z")
> [@sges](#): > > `java.lang.IllegalStateException: Message not fully read (response) for requestId [225], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.transport.TransportService$6@2d61b747], error [false]; resetting` That normally indicates a bug. Would you use `tcpdump` to capture the network traffic on this node and share it with me? We'd need to see the corresponding logs too to pinpoint the problematic message. I can set up a private upload link if you'd rather not share this in public. ---
**Author:** ![sges](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@sges](https://discuss.elastic.co/u/sges)\ **Post date:** [October 15, 2020, 12:22pm UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/3 "2020-10-15T12:22:15Z")
Thanks David. Unfortunately we can't use tcpdump in our secured environment. I'm checking with IT if there is any alternative. Now, I'll be able to share the debugs logs that we captured, can you please share an upload link? Here is another exception that we see consistently in our setup, ``` [2020-10-15T16:07:27,908][WARN][o.e.t.TcpTransport] [10.123.234.2] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.123.234.2:9300, remoteAddress=/10.123.234.1:49996}], closing connection java.lang.IllegalStateException: transport not ready yet to handle incoming requests at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.2.jar:7.9.2] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final] at java.lang.Thread.run(Thread.java:832) [?:?] [2020-10-15T16:07:27,954][WARN][o.e.t.TcpTransport] [10.123.234.2] exception caught on transport layer [Netty4TcpChannel{localAddress=/10.123.234.2:9300, remoteAddress=/10.123.234.1:49998}], closing connection java.lang.IllegalStateException: transport not ready yet to handle incoming requests at org.elasticsearch.transport.TransportService.onRequestReceived(TransportService.java:943) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:136) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:93) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:78) ~[elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:692) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:142) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:117) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) [transport-netty4-client-7.9.2.jar:7.9.2] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final] at java.lang.Thread.run(Thread.java:832) [?:?] ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 15, 2020, 12:49pm UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/4 "2020-10-15T12:49:51Z")
Without the network capture the logs aren't much help I'm afraid. As an alternative, try these trace loggers: ```auto logger.org.elasticsearch.transport.TransportLogger: TRACE logger.org.elasticsearch.transport.netty4.ESLoggingHandler: TRACE ``` Unfortunately this will produce rather a lot of logs, and doesn't always output the information we need, but it might work. You only need keep these loggers enabled until you see a `Message not fully read` message. Do you get `transport not ready yet to handle incoming requests` on an ongoing basis or is it just when the node is starting up? ---
**Author:** ![sges](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@sges](https://discuss.elastic.co/u/sges)\ **Post date:** [October 21, 2020, 9:06am UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/5 "2020-10-21T09:06:58Z")
David, I want to share you the trace logs on this issue, can you share me a location where I can upload them? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 21, 2020, 9:34am UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/6 "2020-10-21T09:34:29Z")
Sure, I've sent you a link in a private message. ---
**Author:** ![sges](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@sges](https://discuss.elastic.co/u/sges)\ **Post date:** [October 21, 2020, 10:40am UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/7 "2020-10-21T10:40:17Z")
We modified our elasticsearch configuration slightly from what I have shared above. And in this cluster setup we deploy a custom plugin to support search in our application and that's upgraded from ES v6.8.3 to ES v7.9.2. Now when we hit search we are seeing exceptions on elasticsearch nodes. There is no response and the nodes are going down in the cluster. Is there a limit on the file size that we can upload in the link you provided? In the uploaded files, I have included all the config files and logs from the respective nodes. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 21, 2020, 11:10am UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/8 "2020-10-21T11:10:59Z")
This looks like a bug, but not in Elasticsearch itself. The broken message has type `indices:data/read/join-search/filterquery[s]` which isn't a native Elasticsearch message so it must be coming from a plugin or other modification. The logs indicate: ```auto [2020-10-21T12:59:22,404][INFO][o.e.p.PluginsService] [10.125.247.49] loaded plugin [join-search-plugin] ``` I suspect removing this plugin will fix things. * * * I don't know if there's a size limit on uploads to the link I shared; if there is one it's at least 32GB since we use this service for heap dumps. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 18, 2020, 11:11am UTC](https://discuss.elastic.co/t/need-help-in-v7-9-2-cluster-configuration-of-3-nodes/252101/9 "2020-11-18T11:11:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.