Elasticsearch plugin

cecoke · December 12, 2018, 1:49am

Hi,

I built a custom search plugin on Elasticsearch and it works totally fine on single node. But when I try to run it on a cluster of nodes, during search query time I got the following error from remote nodes.

java.lang.IllegalStateException: unexpected byte [0x02]
at org.elasticsearch.common.io.stream.StreamInput.readBoolean(StreamInput.java:438) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.common.io.stream.StreamInput.readBoolean(StreamInput.java:428) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.search.builder.SearchSourceBuilder.(SearchSourceBuilder.java:225) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.common.io.stream.StreamInput.readOptionalWriteable(StreamInput.java:759) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.search.internal.ShardSearchLocalRequest.innerReadFrom(ShardSearchLocalRequest.java:213) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.search.internal.ShardSearchTransportRequest.(ShardSearchTransportRequest.java:67) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.transport.RequestHandlerRegistry.newRequest(RequestHandlerRegistry.java:56) ~[elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1568) [elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1442) [elasticsearch-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:63) [transport-netty4-7.0.0-alpha1-SNAPSHOT.jar:7.0.0-alpha1-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.16.Final.jar:4.1.16.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.16.Final.jar:4.1.16.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.16.Final.jar:4.1.16.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) [netty-handler-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:545) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.1.16.Final.jar:4.1.16.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.16.Final.jar:4.1.16.Final]
at java.lang.Thread.run(Thread.java:844) [?:?]

Which classes should I look into for this type of errors?

TimV · December 12, 2018, 4:34am

Without knowing exactly which snapshot build you're using, I can't pin down the exact part of the code wherer this is happening, but I assume it's here:

github.com

elastic/elasticsearch/blob/03daad9812b6469941e5597edd3060e2c78f3a79/server/src/main/java/org/elasticsearch/search/builder/SearchSourceBuilder.java#L225


        docValueFields = null;
    }
}
storedFieldsContext = in.readOptionalWriteable(StoredFieldsContext::new);
from = in.readVInt();
highlightBuilder = in.readOptionalWriteable(HighlightBuilder::new);
indexBoosts = in.readList(IndexBoost::new);
minScore = in.readOptionalFloat();
postQueryBuilder = in.readOptionalNamedWriteable(QueryBuilder.class);
queryBuilder = in.readOptionalNamedWriteable(QueryBuilder.class);
if (in.readBoolean()) {
    rescoreBuilders = in.readNamedWriteableList(RescorerBuilder.class);
}
if (in.readBoolean()) {
    scriptFields = in.readList(ScriptField::new);
}
size = in.readVInt();
if (in.readBoolean()) {
    int size = in.readVInt();
    sorts = new ArrayList<>();
    for (int i = 0; i < size; i++) {

The likely implication is that the serialization code related to the line above that (so, line 224) is the problem. That is, it didn't read the right number of bytes from the stream, so the readBoolean() ended up reading something that isn't actually a boolean.

My guess is that your search pugin has a QueryBuilder with a bug in its serialization. I assume you're extending AbstractQueryBuilder, in which case there are 3 main serialization methods you need to implement:

A constructor that takes (StreamInput in)
The doWriteTo(StreamOutput out) method.
The getWriteableName method.

Some first places to check:

The constructor must call super(in)
The doWriteTo method must write exactly the same fields types, in exactly the same order that the constructor reads them.
Your getWriteableName() method needs to return a name that is unique for each query builder class.

cecoke · December 12, 2018, 10:00pm

Thank you so much, Tim! The issue is fixed now following your suggestion. You great knowledge is much appreciated!

system · January 9, 2019, 10:00pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
What does 'IllegalStateException [unexpected byte' error on transport client denotes? Elasticsearch	3	1468	October 1, 2019
Is it error Version ？, i cannot use GeoDistanceQueryBuilder Elasticsearch	3	668	June 12, 2018
When using elasticsearch-hadoop plugin, IllegalStateException occured Elasticsearch	1	321	July 6, 2017
Why am I getting this `unexpected byte [0x10]` in the constructor for a custom PluginBuilder? Elasticsearch	2	1286	November 8, 2018
java.lang.IllegalStateException: unexpected byte [0x02] Elasticsearch	2	638	July 6, 2020

Elasticsearch plugin

Related topics