Elasticsearch-java - BulkOperation and IndexRequest have different behaviour with same custom JsonpSerializer

[fmarazita]

Hi,

It seems that BulkOperation and IndexRequest with same tDocumentSerialization have different behaviour.
Elastic version: 7.16.3

I created a test with the same upsert operation, with and without using bulk operation. The business logic is very simple:

public IndexRequest<B> indexRequest(String index, B bean) {
        CustomSerializer<D, B, K> serialize = new CustomSerializer<>(...); //-> where CustomSerializer implements JsonpSerializer<B>
       return new IndexRequest.Builder<B>()
                    .index(index)
                    .id(createElasticId(bean))
                    .tDocumentSerializer(serialize)
                    .document(bean)
                    .build();
    }

public BulkOperation bulkIndexRequest(String index, B bean){
     CustomSerializer<D, B, K> serialize = new CustomSerializer<>(...); //-> where CustomSerializer implements JsonpSerializer<B>
    return new BulkOperation.Builder()
            .index(new IndexOperation.Builder<B>()
                    .index(index)
                    .id(createElasticId(bean))
                    .tDocumentSerializer(serialize)
                    .document(bean)
                    .build())
            .build();
}

public BulkResponse sendBulkRequest(List<BulkOperation> indexRequestList) {
   return restClient.bulk(new BulkRequest.Builder().operations(indexRequestList).build()) //--> restClient is a ElasticsearchClient instance
}

public IndexResponse sendIndexRequest(IndexRequest<B> indexRequest) {
    return restClient.index(indexRequest); //--> restClient is a ElasticsearchClient instance
}

In the first case the serialization works, the client sent the serialized document to elastic and I am happy.
The second one failed, No serializer found for class:

Caused by: jakarta.json.JsonException: Jackson exception at co.elastic.clients.json.jackson.JacksonUtils.convertException(JacksonUtils.java:39) at co.elastic.clients.json.jackson.JacksonJsonpMapper.serialize(JacksonJsonpMapper.java:88) at co.elastic.clients.transport.rest_client.RestClientTransport.writeNdJson(RestClientTransport.java:237) at co.elastic.clients.transport.rest_client.RestClientTransport.writeNdJson(RestClientTransport.java:234) at co.elastic.clients.transport.rest_client.RestClientTransport.prepareLowLevelRequest(RestClientTransport.java:211) at co.elastic.clients.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:145) at co.elastic.clients.elasticsearch.ElasticsearchClient.bulk(ElasticsearchClient.java:310) at it.list.lookout.lka.elastic.ElasticDAO.sendBulkRequest(ElasticDAO.java:240) ... 55 more Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: **No serializer found for class** ... and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain:

Maybe I am wrong but there is something strange on on method BulkRequest _serializables.

Thanks in advance
Fabrizio

[fmarazita]

Any news?

[swallez (Sylvain Wallez)]

There is indeed an issue with IndexOperation which is serialized in a specific way (bulk requests are sent as nd-json with operation metadata and document sent as separate json documents) and it effectively does not use tDocumentSerializer. We're going to fix this.

However, the general recommendation would be to avoid using these tDocumentSerializer properties and let Jackson's object mapper handle serialization of your application objects. Providing your own serializer involves some rather low details the Java client's underlying framework that in practice can be easily handled by a regular JSON object mapper as Jackson. We're actually considering deprecating these serializer properties and remove them at some point.

That being said, the exception shows that Jackson doesn't know how to serialize your bean because it cannot fin any properties. You should investigate in this direction so that you don't need to set tDocumentSerializer.