"failed to check serialization" errors for SearchPlugin

[Daniel_Penning]

I am currently working on upgrading an ES Plugin from 2.3 to 5.2. When running my unit tests i randomly get RuntimeExceptions when executing searches that use my own QueryBuilder. It looks like somehow the NamedWriteableRegistry can sometimes not find my QueryBuilder to read it from serialized form.

Running exactly the same query multiple times does work after some tries though. The version changes randomly every time and i have seen all versions from 5.0.0 to 5.2.1.

Excample for such an exception:

java.lang.RuntimeException: failed to check serialization - version [5.0.1] for streamable [SearchRequest{searchType=DFS_QUERY_THEN_FETCH, indices=[test], indicesOptions=IndicesOptions[id=38, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_alisases_to_multiple_indices=true, forbid_closed_indices=true], types=[test], routing='null', preference='null', requestCache=null, scroll=null, source={
  "query" : {
    "condition" : {
      "cond" : {
        "a" : 1
      }
    }
  },
  "_source" : {
    "includes" : [ ],
    "excludes" : [ ]
  },
  "stored_fields" : "name"
}}]
        at org.elasticsearch.common.io.stream.NamedWriteableRegistry.getReader(NamedWriteableRegistry.java:112)
        at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:45)
        at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:39)
        at org.elasticsearch.common.io.stream.StreamInput.readOptionalNamedWriteable(StreamInput.java:850)
        at org.elasticsearch.search.builder.SearchSourceBuilder.<init>(SearchSourceBuilder.java:197)
        at org.elasticsearch.common.io.stream.StreamInput.readOptionalWriteable(StreamInput.java:709)
        at org.elasticsearch.action.search.SearchRequest.readFrom(SearchRequest.java:320)
        at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertVersionSerializable(ElasticsearchAssertions.java:677)
        at org.elasticsearch.test.transport.AssertingLocalTransport.sendRequest(AssertingLocalTransport.java:106)
        at org.elasticsearch.transport.local.LocalTransport$1.sendRequest(LocalTransport.java:216)
        at org.elasticsearch.transport.TransportService.sendRequestInternal(TransportService.java:554)
        at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:485)
        at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:473)
        at org.elasticsearch.action.TransportActionNodeProxy.execute(TransportActionNodeProxy.java:51)
        at org.elasticsearch.client.transport.TransportProxyClient.lambda$execute$0(TransportProxyClient.java:59)
        at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247)
        at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59)
        at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:356)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403)
        at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80)
        at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54)
        at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:62)
        at de.stroeermediabrands.elasticsearch.condition.test.AbstractIntegrationTestCase.checkQuery(AbstractIntegrationTestCase.java:85)
        at de.stroeermediabrands.elasticsearch.condition.test.ConditionTestCase.numberListCondition(ConditionTestCase.java:75)

[cbuescher (Christoph)]

Hi,

did you register your new "condition" QueryBuilder with the NamedWritableRegistry in your plugin? I believe the way to do it in 5.2 is to overwrite the Plugin#getNamedWriteables() method and return a List of NamedWriteableRegistry.Entry instances representing your query. For QueryBuilder I think you need to provide the class that all query builders are grouped under (QueryBuilder.class), a unique name and a method that creates a new QueryBuilder by reading from an input stream. Something like this, but with your query builders specs of course:

  @Override
  public List<NamedWriteableRegistry.Entry> getNamedWriteables() {
        return Collections.singletonList(new NamedWriteableRegistry.Entry(QueryBuilder.class, MatchAllQueryBuilder.NAME, MatchAllQueryBuilder::new));
  }

I'm not 100% sure but on 5.2 you also might have to register the parser by overwriting Plugin#getNamedXContent() in a slightly similar way. In this case I think this would work (of course adapted to your case), but I'm not entirely sure and haven't tested this:

    @Override
    public List<NamedXContentRegistry.Entry> getNamedXContent() {
        ContextParser<Object, ? extends Optional> parser = (XContentParser p, Object c) -> MatchAllQueryBuilder.fromXContent((QueryParseContext) c);
        NamedXContentRegistry.Entry e = new NamedXContentRegistry.Entry(Optional.class, new ParseField(MatchAllQueryBuilder.NAME), parser);
        return Collections.singletonList(e);
    }
 

Maybe @nik9000 can confirm if this should be working or not?

[Daniel_Penning]

I implemented org.elasticsearch.plugins.SearchPlugin and used the following method to register my QueryBuilder.

@Override
public List<QuerySpec<?>> getQueries()
{
	QuerySpec<?> querySpec = new QuerySpec<>(ConditionQueryBuilder.NAME, ConditionQueryBuilder::new, ConditionQueryBuilder::fromXContent);
	return Collections.singletonList(querySpec);
}

I tried using the two methods you suggested instead but got exactly the same exceptions.
The class org.elasticsearch.search.SearchModule already takes care of registering in the NamedWriteableRegistry and NamedXContentRegistry based on the QuerySpecs given by the available SearchPlugins.

[cbuescher (Christoph)]

Yes, what you say makes sense, I didn't know the SearchPlugin interface so far.
I'm still wondering if the new query is registered correctly since the stack trace (NamedWriteableRegistry.java:112) points to the lookup of the particular reader by its name.

The version changes randomly because test setup you are using is using AssertingLocalTranspost which runs ElasticsearchAssertions.assertVersionSerializable() on each request. How is your unit tests setup? I see AbstractIntegrationTestCase in the stack trace, is that class derived from one of the classes in the elasticsearch test framework?

[Daniel_Penning]

Yes, i am using the elasticsearch test framework. The class AbstractIntegrationTestCase is derived from org.elasticsearch.test.ESIntegTestCase.

I had a closer look at which client settings are randomized and which values caused exceptions to be thrown. My tests worked fine when the REST client is used but not with the transport client. So i had a closer look at the ESIntegTestCase and found my problem. I did only register my plugin in ESIntegTestCase#nodePlugins(). But as the transport client expects plugins to be registered in ESIntegTestCase#transportClientPlugins() it didn't know anything about my plugin.

Thanks for your help.

[cbuescher (Christoph)]

And thanks for sharing your solution!