BulkRequest failed when handle JSON data

Elastic Stack Elasticsearch
1

Hi all,

We want to index data into the ES with bulk api in the java client, the dependency is:
elasticsearch-java: 8.13.2

the source data is a JSON string like this:

t      String createDoc1 = "{"
              + "            \"learning_item_rating\": 4.5,"
              + "            \"learning_item_title\": \"Java 101\","
              + "            \"learning_item_classification\": \"Program\","
              + "            \"learning_item_rating_count\": 10,"
              + "            \"learning_item_id\": \"001\""
              + "        }";

JsonpMapper jsonpMapper = ES_WRITE_CLIENT._transport().jsonpMapper();
              JsonProvider jsonProvider = jsonpMapper.jsonProvider();
              JsonData doc = JsonData.from(jsonProvider.createParser(new StringReader(createDoc1)), jsonpMapper);
              bulkBuilder.operations(op->op.create(item -> item.index("my_index").id("dod_id").document(doc)));

But the request failed with this error:
ErrorCause: {"type":"document_parsing_exception","reason":"[1:24] object mapping for [learning_item_title] tried to parse field [learning_item_title] as object, but found a concrete value"}.

We tried with this solution:

bulkBuilder.operations(op->op.create(item -> item.index("my_index").withJson(new StringReader(createDoc1))));

but this solution failed because of this error:
java.lang.IllegalArgumentException: Class class co.elastic.clients.elasticsearch.core.bulk.CreateOperation cannot be read from JSON
at co.elastic.clients.util.WithJsonObjectBuilderBase.withJson(WithJsonObjectBuilderBase.java:46)

any ideas about how to fix this issue?

thanks

2

Look at the mappings for the index you are trying to index into using the get mapping API. It seems the field learning_item_title, possibly due to earlier documents or a static mapping, is mapped as an object. This means that trying to index a string for this field results in a mapping conflict.

3

this is the learning_item_title's mapping setup:

{
  "mappings": {
    "dynamic": "strict",
    "properties": {
    "learning_item_title": {
        "properties": {
          "de_DE": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "doc_values": false,
                "normalizer": "lowercase"
              },
              "sayt": {
                "type": "search_as_you_type",
                "doc_values": false,
                "max_shingle_size": 3,
                "analyzer": "rebuilt_german_delimiter"
              },
              "sort": {
                "type": "icu_collation_keyword",
                "index": false,
                "language": "de",
                "country": "DE"
              }
            },
            "analyzer": "rebuilt_german_delimiter",
            "index_prefixes": {
              "min_chars": 2,
              "max_chars": 19
            }
          },
          "en_US": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "doc_values": false,
                "normalizer": "lowercase"
              },
              "sayt": {
                "type": "search_as_you_type",
                "doc_values": false,
                "max_shingle_size": 3,
                "analyzer": "rebuilt_english_delimiter"
              },
              "sort": {
                "type": "icu_collation_keyword",
                "index": false,
                "language": "en",
                "country": "US"
              }
            },
            "analyzer": "rebuilt_english_delimiter",
            "index_prefixes": {
              "min_chars": 2,
              "max_chars": 19
            }
          }}}}}}
4

That is as expected. It has fields specified underneath, e.g. de_DE and en_US, which means that it is an object and can not take strings. You will need to change the structure of the document you try to index.

1 Like
5

got it, let me try again.

By the way, does the bulkrequest support the script type under the Update operation? If so, is there any documents about how to setup script content in the bulkrequest (the source data still come from JSON like:'{'script:'xxxxx'}')?