Update by query don't accept user-defined param in java high level client

Elastic Stack Elasticsearch
1

Hi,

I'm using REST High Level Client in v7.5
I'm getting error when I tried to update all document by UpdateByQueryRequest.
Error message : cannot write xcontent for unknown value of type class entity.Navigation

Belows are my sample code.

private static void updateByQueryForNavigation() throws IOException {
    Navigation navi1 = new Navigation(Arrays.asList("1", "2", "3"), "2020-01-01T10:00:00", "2020-01-01T10:00:00");
    Navigation navi2 = new Navigation(Arrays.asList("4", "5", "6"), "2020-01-01T10:00:00", "2020-01-01T10:00:00");
    List<Navigation> naviList = new ArrayList<>();
    naviList.add(navi1);
    naviList.add(navi2);

    Map<String, Object> params = new HashMap<>();
    params.put("navigation", naviList);

    String source = "ctx._source.navigation = params.navigation";
    UpdateByQueryRequest request = new UpdateByQueryRequest("test");
    request.setScript(new Script(ScriptType.INLINE, "painless", source, params));

    client.updateByQuery(request, RequestOptions.DEFAULT);
}

=========================================================================

package entity;

import java.io.Serializable;
import java.util.List;

public class Navigation implements Serializable {
    private static final long serialVersionUID = 1L;

    private List<String> navigationCode;
    private String startDate;
    private String endDate;

    public Navigation(List<String> navigationCode, String startDate, String endDate) {
        this.navigationCode = navigationCode;
        this.startDate = startDate;
        this.endDate = endDate;
    }

    public Navigation() {
    }

    public List<String> getNavigationCode() {
        return navigationCode;
    }

    public String getStartDate() {
        return startDate;
    }

    public String getEndDate() {
        return endDate;
    }

}

If I convert naviList to json then set it into param, error doesn't happen but result is not what I expected. Json string is directly insert into document. My expectation is naviList is inserted as nested array object.
Before

"hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "test name"
        }
      }
    ]

After(expectation)

"hits" : [
  {
    "_index" : "test",
    "_type" : "_doc",
    "_id" : "1",
    "_score" : 1.0,
    "_source" : {
      "navigation" : [
        {
          "endDate" : "2020-01-01T10:00:00",
          "navigationCode" : [
            "1",
            "2",
            "3"
          ],
          "startDate" : "2020-01-01T10:00:00"
        },
        {
          "endDate" : "2020-01-01T10:00:00",
          "navigationCode" : [
            "4",
            "5",
            "6"
          ],
          "startDate" : "2020-01-01T10:00:00"
        }
      ],
      "name" : "test name"
    }
  }
]

Below request works as expected. I just would like to achieve same output by Java High Level client.

POST /test/_update_by_query
{
  "script" : {
    "source" : "ctx._source.navigation = params.navigation",
    "params" : {
      "navigation" : [{"navigationCode":["1","2","3"],"startDate":"2020-01-01T10:00:00","endDate":"2020-01-01T10:00:00"},{"navigationCode":["4","5","6"],"startDate":"2020-01-01T10:00:00","endDate":"2020-01-01T10:00:00"}]
    },
    "lang" : "painless"
  }

}

I already checked below discussion but couldn't find answer.

Thank you in advance.
Tsubasa

1 Like
2

You can not directly send a JavaBean to Elasticsearch. You need first to serialize it to json.
You can use Jackson for that.

An example here:

3

Hi,
Thank you for your quick reply.

I changed my code like below but result wasn't what I expected. byte chars are directly inserted.
How can I config updateByQueryRequest so that it can handle byte array data?

private static void updateByQueryForNavigation() throws IOException {
        Navigation navi1 = new Navigation(Arrays.asList("1", "2", "3"), "2020-01-01T10:00:00", "2020-01-01T10:00:00");
        Navigation navi2 = new Navigation(Arrays.asList("4", "5", "6"), "2020-01-01T10:00:00", "2020-01-01T10:00:00");
        List<Navigation> naviList = new ArrayList<>();
        naviList.add(navi1);
        naviList.add(navi2);

        ObjectMapper mapper = new ObjectMapper();
        byte[] bytes = mapper.writeValueAsBytes(naviList);

        Map<String, Object> params = new HashMap<>();
        params.put("navigation", bytes);

        String source = "ctx._source.navigation = params.navigation";
        UpdateByQueryRequest request = new UpdateByQueryRequest("test");
        request.setScript(new Script(ScriptType.INLINE, "painless", source, params));

        client.updateByQuery(request, RequestOptions.DEFAULT);
    }
4

Could you try with writeValueAsString instead and log debug the generated String?

5

Hi,
I tried writeValueAsString but json string was directly inserted like below.
There was no error log. This is very weird because If I set same json in params in Kibana like I my first post, it works perfectly. If you need additional information let me know.

"hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "navigation" : """{"navigationCode":["1","2","3"],"startDate":"2020-01-01T10:00:00","endDate":"2020-01-01T10:00:00"}""",
          "name" : "test name"
        }
      }
    ]
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

7

What happens if you print the value generated by writeValueAsString?

Like:

String json = mapper.writeValueAsString(naviList);
logger.info(json);
9

Having this same issue - Serialized json looks fine before sending as a parameter in the UpdateByQuery request, but get an error on the way back

String serialized = objectMapper.writeValueAsString(object);
Map<String, Object> params = ImmutableMap.<String, Object>builder().put("updatedObject", serializedObject).build();
new Script(ScriptType.INLINE, "painless","ctx._source.oldField = params.updatedObject ",
    params
    );

When printing this query:

update-by-query [shared] updated with Script{type=inline, lang='painless', idOrCode='ctx._source.oldField = params.updatedObject ', options={}, params={updatedObject={"field1":"61582121-d971-4aa8-8db3-c099a688fa20","field2":"17f328af-db2b-4aa7-b6ba-3c203b305e67","field3":"some_data"}}}

Got this exception: "cause":{"type":"mapper_parsing_exception","reason":"object mapping for [brandInfo] tried to parse field [brandInfo] as object, but found a concrete value"},"status":400} making it seem like we attempted to write the object as a string value. I have verified that the type matches the mapping of the underlying index (oldField and newObject have the same json shape)

I imagine it'd be helpful if Script had a constructor allowing you to specify XContentType, similar to the Index/Update queries (.source(bytes, XContentType.JSON))

Note that this works when manually creating the map, but it'd be great to be able to leverage Jackson serializers for complex/nested data types for these update scripts.

Elasticsearch verison - 7.9.1

10

Right now I'm putting in a hack in which I expect certain indexed properties to implement a Map<String, Object> toMap() but this will break when moving to deeper nested objects

11

Found a solution for this - Use XContentFactory to convert the serialized string to a map as the Script expects:

String serializedObj = DocumentMapper.serializeObject(complexJavaPojo, objectMapper);
Map<String, Object> serializedParams =
    XContentHelper.convertToMap(
        XContentFactory.xContent(XContentType.JSON), serializedObj, false);

return new Script(
    ScriptType.INLINE,
    SCRIPT_LANG,
    String.format("ctx._source.%s = params", targetFieldPath),
    serializedParams);
2 Likes