"include_in_root" option with wildcard field causes "illegal_argument_exception"

Tomo_M · November 28, 2021, 6:49am

Hello,
I tried to use "include_in_root" option with nesting wildcard field but got error.

The Elasticsearch version is as follows:

"version" : {
    "number" : "7.15.2",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "93d5a7f6192e8a1a12e154a2b81bf6fa7309da0c",
    "build_date" : "2021-11-04T14:04:42.515624022Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"

First I created an index:

PUT test_nested_field
{
  "settings":{},
  "mappings":{
    "properties": {
      "parent_field": {
        "properties": {
          "nested_field": {
            "type": "nested",
            "include_in_root": true,
            "properties": {
              "wildcard_field": {
                "type": "wildcard"
              },
              "float_field": {
                "type": "float"
              },
              "integer_field": {
                "type": "integer"
              }
            }
          }
        }
      }
    }
  }
}

Then the following index was created successfully.

{
  "test_nested_field" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "parent_field" : {
          "properties" : {
            "nested_field" : {
              "type" : "nested",
              "include_in_root" : true,
              "properties" : {
                "float_field" : {
                  "type" : "float"
                },
                "integer_field" : {
                  "type" : "integer"
                },
                "wildcard_field" : {
                  "type" : "wildcard"
                }
              }
            }
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "number_of_shards" : "1",
        "provided_name" : "test_nested_field",
        "creation_date" : "1638079974223",
        "number_of_replicas" : "1",
        "uuid" : "cypT5YzJTCOwOMLZnfM-ww",
        "version" : {
          "created" : "7150299"
        }
      }
    }
  }
}

But when I tried to index a document with the following query,

POST test_nested_field/_doc
{
  "parent_field": {
    "nested_field": [
      {
        "wildcard_field": "aaa",
        "integer_field": "14",
        "float_field": "0.52"
      },
      {
        "wildcard_field": "bbb",
        "integer_field": "1",
        "float_field": "0.04"
      }
    ]
  }
}

I got the following error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "DocValuesField \"parent_field.nested_field.wildcard_field\" appears more than once in this document (only one value is allowed per field)"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "DocValuesField \"parent_field.nested_field.wildcard_field\" appears more than once in this document (only one value is allowed per field)"
  },
  "status" : 400
}

Curiously the whole process went well

with a single nesting object,
without "include_in_root" option or
without nesting wildcard field.

With a single nesting object:

POST test_nested_field/_doc
{
  "parent_field": {
    "nested_field": [
      {
        "wildcard_field": "aaa",
        "integer_field": "14",
        "float_field": "0.52"
      }
    ]
  }
}

{
  "_index" : "test_nested_field",
  "_type" : "_doc",
  "_id" : "VnVKZX0Bxutvf9KVhJFP",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}

Without "include_in_root" option:

PUT test_nested_field_without_include_in_root
{
  "settings":{},
  "mappings":{
    "properties": {
      "parent_field": {
        "properties": {
          "nested_field": {
            "type": "nested",
            "properties": {
              "wildcard_field": {
                "type": "wildcard"
              },
              "float_field": {
                "type": "float"
              },
              "integer_field": {
                "type": "integer"
              }
            }
          }
        }
      }
    }
  }
}

POST test_nested_field_without_include_in_root/_doc
{
  "parent_field": {
    "nested_field": [
      {
        "integer_field": "14",
        "float_field": "0.52"
      },
      {
        "integer_field": "1",
        "float_field": "0.04"
      }
    ]
  }
}

{
  "_index" : "test_nested_field_without_include_in_root",
  "_type" : "_doc",
  "_id" : "UnU8ZX0Bxutvf9KV-I_t",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}

Without nesting wildcard field:

PUT test_nested_field_without_wildcard
{
  "settings":{},
  "mappings":{
    "properties": {
      "parent_field": {
        "properties": {
          "nested_field": {
            "type": "nested",
            "include_in_root": true,
            "properties": {
              "float_field": {
                "type": "float"
              },
              "integer_field": {
                "type": "integer"
              }
            }
          }
        }
      }
    }
  }
}

POST test_nested_field_without_wildcard/_doc
{
  "parent_field": {
    "nested_field": [
      {
        "integer_field": "14",
        "float_field": "0.52"
      },
      {
        "integer_field": "1",
        "float_field": "0.04"
      }
    ]
  }
}

{
  "_index" : "test_nested_field_without_wildcard",
  "_type" : "_doc",
  "_id" : "N3U8ZX0Bxutvf9KVDY9n",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

I couldn't find any hint in Nested field type and Wildcard field type.
Is there something wrong in my query?

Thanks.

Tomo_M · November 28, 2021, 6:56am

I found a simmilar issue in Github.

github.com/elastic/elasticsearch

Nested geo_* type with doc values and include_in_parent fails with multiple values

opened 03:13PM - 06 Jul 15 UTC

closed 01:26PM - 19 Sep 15 UTC

pickypg

>bug help wanted :Search/Mapping v1.7.1

If you create a `nested` `geo_*` type and you attempt to use both `include_in_pa…rent` and `doc_values`, then you cannot index multiple values. If you remove either setting, then it works. ``` PUT /test { "mappings": { "type": { "properties": { "location": { "type": "nested", "include_in_parent": true, "properties": { "point": { "type": "geo_point", "doc_values": true } } } } } } } # Works PUT /test/type/1 { "location" : { "point" : [0, 0] } } # Fails PUT /test/type/2 { "location" : [ { "point" : [0, 0] }, { "point" : [1, 1] } ] } ``` The first attempt works as expected, but the second attempt fails with an odd exception. ``` json { "error": "IllegalArgumentException[DocValuesField \"location.point\" appears more than once in this document (only one value is allowed per field)]", "status": 500 } ``` This is related to https://github.com/elastic/elasticsearch/issues/10653#issuecomment-94097576, which notes that `include_in_parent` is ignored for `geo_*` types, but it does not suggest that it will fail upon trying to make use of it. This behavior should probably be explicitly blocked. Note: changing `geo_point` to some type that does work with `include_in_parent`, such as `long`, does work.

I'm sorry if my case is the same as the issue.

Tomo_M · November 29, 2021, 6:11am

I came up with a workaround by myself.
It is possible to use "copy_to" option to achieve similar effect.
I write it here to someone who will has the same problem.

PUT test_nested_field
{
  "settings":{},
  "mappings":{
    "properties": {
      "parent_field": {
        "properties": {
          "nested_field": {
            "type": "nested",
            "properties": {
              "wildcard_field": {
                "type": "wildcard",
                "copy_to": "wildcard_field"
              },
              "float_field": {
                "type": "float"
              },
              "integer_field": {
                "type": "integer"
              }
            }
          }
        }
      },
      "wildcard_field":{
        "type": "wildcard"
      }
    }
  }
}

POST test_nested_field/_doc
{
  "parent_field": {
    "nested_field": [
      {
        "wildcard_field": "aaa",
        "integer_field": "14",
        "float_field": "0.52"
      },
      {
        "wildcard_field": "ccc",
        "integer_field": "1",
        "float_field": "0.04"
      }
    ]
  }
}

GET test_nested_field/_search
{
  "query": {
    "wildcard": {
      "wildcard_field":"aa*"
    }
  }
}

Please close the topic.
Thanks!

system · December 27, 2021, 6:12am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.