Weird behaviour on document update to remove value from a field

Elastic Stack Elasticsearch
1

Elasticsearch version 8.13.3

Assume there is a document with id test in index testindex with following fields.

{
  "field1": "any1",
  "field2": "any2"
}

if i execute a update request like
POST: <hostname>/testindex/_update/test?refresh=false
Body:

{
	"doc": {
		"field2": null
	},
	"doc_as_upsert": true
}

The document gets updated to

{
  "field1": "any1",
  "field2": null
}

But the same if we execute via the client, value in the field field2 does not get updated to null.

Map<String,Object> testbody = new HashMap<>();
        testbody.put("field2",null);

UpdateRequest<Map<String,Object>,Map<String,Object>> testRequest = new UpdateRequest.Builder<Map<String,Object>,Map<String,Object>>()
                .id("test")
                .index("testindex")
                .doc(testbody)
                .docAsUpsert(true)
                .build();
client.update(testRequest,HashMap.class);

Any idea why the client behaves like this?

2

Hello!
This is weird, I have tried this with server version 8.14.0 and java client versions 8.13.3 and the latest 8.14.3, and it works as expected, field2 gets updated to null. Could you give me more information on which client + server versions are you using?

3

The server version is 8.13.4 (https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.13.4-darwin-x86_64.tar.gz) and the client version is 8.13.3 (https://mvnrepository.com/artifact/co.elastic.clients/elasticsearch-java/8.13.3).

4

Also just now tried the code against the newer es version both server and java elastic client as 8.14.3

The issue still existed.

Also just to be safe I logged out the request (logger.info("{}"),request) and this was the update request which got generated and executed.

UpdateRequest: POST /testindex/_update/test?refresh=false {"doc":"{field1=any1, field2=null}","doc_as_upsert":true}

But even after this was executed and the index refreshed, the field2 was still present with the same value (any2) in the document.

5

Strange. The way below works but it would be good to understand why your way failed, maybe open an issue.

.doc(JsonData.fromJson(new ObjectMapper().writeValueAsString(testbody)));

6

I wrote a quick reproducer, could you execute this and let me know the output?

Map<String,Object> testbody = new HashMap<>();
testbody.put("field1","any1");
testbody.put("field2","any2");

esClient.index(ix -> ix.index("test-upsert-null").id("test").document(testbody).refresh(Refresh.WaitFor));

SearchResponse<HashMap> res = esClient.search(s -> s.index("test-upsert-null"),HashMap.class);
System.out.println("current field2 value: " + res.hits().hits().get(0).source().get("field2"));

testbody.put("field2",null);

esClient.update(ur -> ur.id("test").doc(testbody).index("test-upsert-null").docAsUpsert(true).refresh(Refresh.WaitFor),HashMap.class);
res = esClient.search(s -> s.index("test-upsert-null"),HashMap.class);
System.out.println("final field2 value: " + res.hits().hits().get(0).source().get("field2"));
1 Like
7

Hey,

this was the output

current field2 value: any2
final field2 value: any2
8

but interestingly, the fix given by @RabBit_BR worked.

Map<String,Object> testbody = new HashMap<>();
        testbody.put("field1","any1");
        testbody.put("field2","any2");

        client.index(ix -> ix.index("test-upsert-null").id("test").document(testbody).refresh(Refresh.WaitFor));

        SearchResponse<HashMap> res = client.search(s -> s.index("test-upsert-null"),HashMap.class);
        System.out.println("current field2 value: " + res.hits().hits().get(0).source().get("field2"));

        testbody.put("field2",null);

        client.update(ur -> {
            try {
                return ur.id("test").doc(JsonData.fromJson(new ObjectMapper().writeValueAsString(testbody))).index("test-upsert-null").docAsUpsert(true).refresh(Refresh.WaitFor);
            } catch (JsonProcessingException e) {
                throw new RuntimeException(e);
            }
        },HashMap.class);
        res = client.search(s -> s.index("test-upsert-null"),HashMap.class);
        System.out.println("final field2 value: " + res.hits().hits().get(0).source().get("field2"));

output

current field2 value: any2
final field2 value: null
1 Like
9

Do you perhaps have any custom configuration on the json mapper used to create the elasticsearch client?

10

Unless I am making a mistake in the below code.
I would say I am not using any custom configuration.

builder = RestClient
                .builder(new HttpHost(this.targetUrl.getHost(), this.targetUrl.getPort(), this.targetUrl.getScheme()));


        RestClient lowLevelRestClient = builder.build();
        transport = new RestClientTransport(
                lowLevelRestClient,
                new JacksonJsonpMapper()
        );
        clientES = new ElasticsearchClient(transport);
11

Great point @ltrotta . @Vijeya_Nidhi , change as below and test your original code.

    JacksonJsonpMapper mapper = new JacksonJsonpMapper();
    mapper.objectMapper().setSerializationInclusion(JsonInclude.Include.ALWAYS);
    ElasticsearchTransport transport = new RestClientTransport(
            restClient, mapper);
1 Like
12

yup the mapper change suggested worked.

output

current field2 value: any2
final field2 value: null

Thanks for the help.

13

Thank you both for the help, yes the issue was with the default JacksonJsonpMapper, by initializing the JacksonJsonpMapper without passing any ObjectMapper, the default used constructor will build:

    public JacksonJsonpMapper() {
        this((new ObjectMapper()).configure(SerializationFeature.INDENT_OUTPUT, false).setSerializationInclusion(Include.NON_NULL));
    }

which ignores null value in the serialization. I assume you got the code from the official documentation, we'll update it with the recommended way to initialize the mapper.

1 Like
14

yup, we are in the process of migrating our code base from high level rest client to the recommended java api client, so for the initialisation purpose we just used the code snippet in the documentation to initialise the client.

but it does makes sense looking back, why until the request is generated, it was having null in the body, but on execution of the request it failed to remove the value from the field.

Any thanks to both of you for the quick help and pointing out where the problem was.

1 Like