Bulk indexing with x-pack Exception

franck.lefebure · July 6, 2017, 11:20am

Hi,
With a 5.4.3 single node cluster and a 5.4.3 java client, last JDK, all centos Linux : I meet the following error when the x-pack is installed on the cluster (with config xpack.security.enabled: false)

org.elasticsearch.transport.TransportSerializationException: Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.state.ClusterStateResponse]
        at org.elasticsearch.transport.TcpTransport.handleResponse(TcpTransport.java:1442) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1414) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-client-5.4.3.jar:5.4.3]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[netty-common-4.1.11.Final.jar:4.1.11.Final]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_131]
Caused by: java.lang.IllegalArgumentException: Unknown NamedWriteable [org.elasticsearch.cluster.metadata.MetaData$Custom][ml]
        at org.elasticsearch.common.io.stream.NamedWriteableRegistry.getReader(NamedWriteableRegistry.java:112) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:45) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:39) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.cluster.metadata.MetaData.readFrom(MetaData.java:675) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.cluster.ClusterState.readFrom(ClusterState.java:659) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.action.admin.cluster.state.ClusterStateResponse.readFrom(ClusterStateResponse.java:58) ~[elasticsearch-5.4.3.jar:5.4.3]
        at org.elasticsearch.transport.TcpTransport.handleResponse(TcpTransport.java:1439) ~[elasticsearch-5.4.3.jar:5.4.3]
        ... 23 common frames omitted

This problem should be related to Remote Transport Exception in 5.1.1 w/JDK 1.8.0_111

Franck

spinscale · July 6, 2017, 3:01pm

Hey,

can you (for testing purposes) follow the steps to use a PreBuiltXPackTransportClient at https://www.elastic.co/guide/en/x-pack/5.4/java-clients.html#transport-client and check if that works? I could imagine, that with ML there are some specialties, that need to be taken into account when parsing the cluster state.

--Alex

franck.lefebure · July 6, 2017, 11:53pm

Hi Alexander,

With the PreBuiltXPackTransportClient, I don't meet no more the Exception..

So can I have for conclusion that, with the x-pack plugin installed on the server, java client MUST be a PreBuiltXPackTransportClient, in every cases.
(In my case I just wanted x-pack for the monitoring "a la Marvel" features) ?

Thanks, whatever.

Franck

dadoonet · July 7, 2017, 5:02am

Very interesting to know. Thanks for opening this thread!

spinscale · July 7, 2017, 7:01am

Hey,

you could also try to set xpack.ml.enabled: false in the settings that you pass to the TransportClient and see if that solves the issue.

--Alex

system · August 4, 2017, 7:01am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.