IllegalArgumentException - DocValuesField appears more than once in this document

Elastic Stack Elasticsearch
1

I am seeing a strange problem. I am running Elasticsearch version 1.7.5. I have created a mapping:

{
   "ali-development": {
      "mappings": {
         "lead": {
            "dynamic": "false",
            "properties": {
               ...several fields omitted for brevity...
               "custom_fields": {
                  "type": "nested",
                  "include_in_root": true,
                  "properties": {
                     "custom_field_definition_id": {
                        "type": "long",
                        "doc_values": true,
                        "include_in_all": false
                     },
                     "float_value": {
                        "type": "float",
                        "doc_values": true,
                        "include_in_all": false
                     },
                     "integer_value": {
                        "type": "integer",
                        "doc_values": true,
                        "include_in_all": false
                     },
                     "option_id": {
                        "type": "long",
                        "doc_values": true,
                        "include_in_all": false
                     },
                     "string_value": {
                        "type": "string",
                        "index": "not_analyzed",
                        "doc_values": true,
                        "include_in_all": false
                     },
                     "string_value_espp_downcase": {
                        "type": "string",
                        "index": "not_analyzed",
                        "doc_values": true,
                        "include_in_all": false
                     }
                  }
               }
            }
         }
      }
   }
}

In my automated test suite, in the middle of a bulk indexing opertion, I am attempting to insert a document with multiple custom_field_definition_ids:

{
    ...more data omitted...
    "custom_fields": [
      {
        "string_value": null,
        "id": 7,
        "custom_field_definition_id": 2,
        "integer_value": null,
        "float_value": 3.0,
        "option_id": null,
        "string_value_espp_downcase": null
      },
      {
        "string_value": null,
        "id": 8,
        "custom_field_definition_id": 3,
        "integer_value": 1442871695,
        "float_value": null,
        "option_id": null,
        "string_value_espp_downcase": null
      },
      {
        "string_value": null,
        "id": 9,
        "custom_field_definition_id": 4,
        "integer_value": null,
        "float_value": 1.0,
        "option_id": null,
        "string_value_espp_downcase": null
      }
    ]
  }

On one specific environment, this produces an error:
{
"index": {
"_index": "ali-test",
"_type": "lead",
"_id": "entity_type:21/entity_id:6",
"status": 500,
"error": "IllegalArgumentException[DocValuesField "custom_fields.custom_field_definition_id" appears more than once in this document (only one value is allowed per field)]"
}
},

Nothing in the documentation for doc_values says anything about requiring values in a doc_values field to be unique. Also, it's really strange that this error is only happening on one particular environment. I can create this document on other environments without any problems at all. What's going on? Is this a symptom of an underlying problem? Any way I can get some better diagnostic info?

2

Hi @mpowers

I can only recreate this issue on an index created before Elasticsearch 1.4.0. Back then, numeric doc values didn't exist and we used binary doc values instead. This example works just fine on indices created from 1.4 onwards.