How to handle dot in field names in ES 7.0

Elastic Stack Elasticsearch
1

I was recently hit by the fieldname-with-dots issue in ES. I search on google but found the answer for ES 2.X version while for other version none of the option work for me.

execution.command = /bin/bash
execution.command.args = emrStatusCluster

"Note : type_name is deprecated for ES 7.0 so default type is _doc "

I use curl -XPOST localhost:9200/index_name/_doc -H "Content-Type: application/json" -d @jsondata.json to load data and get following mssg on the console.

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't merge a non object mapping [execution.command] with an object mapping [execution.command]"}],"type":"illegal_argument_exception","reason":"Can't merge a non object mapping [execution.command] with an object mapping [execution.command]"},"status":400

I tried dot_expander but didnot find anything much useful and constructive.

Thanks in Advance ,
Shanky

2

What do you have in jsondata.json?

3

A single Json record in which execution.command and execution.command.argrs are present somewhere as key.

4

You need to fix its content probably.

5

I don't have any control on content since content comes from nifi where i call elastic search api is there any way to deal as it is given for ES 2.X version where we can allow dot in field name.

Thanks in advance.

6

I don't really know as I can't see the content.

7

Ok let me give you small view of Content which is in json format:

{  
   "s3.version":"mc5ETuM",
   "invokehttp.tx.id":"f75283d92c",
   "sqs.receipt.handle":"Zl/yWciVceGIiWiC5sNkQ=",
   "Server":"J4)",
   "s3.etag":"712a41",
   "listEMResponse":"",
   "indexed_file_path":"fasfd",
   "schemaName":"",
   "uuid":"cf",
   "sqs.message.id":"ae",
   "sqs.SenderId":"AI",
   "hash.value":"7b77",
   "tableName":"Table1",
   "execution.command":"/bin/bash",
   "filename":"1554448956907.csv",
   "s3.key":"/15544_TB_.csv",
   "execution.command.args":".-1"
}

Now the problem is execution.command.args and execution.command both contain data if i make execution.command type as object then on running it will give error what i did is :

PUT index_try
{
  "mappings": {
    "json": {
      "properties": {
        "created_at": {
          "type": "date",
          "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
        },
        "execution": {
          "type": "object",
          "properties": {
            "command": {
              "type": "object",
              "properties": {
                "args": {
                  "type": "text"
                }
              }
            }
          }
        }
      }
    }
  },
  "settings": {
    "number_of_shards": 6,
    "number_of_replicas": 1
  }
}

I refered following post may be this help you :slight_smile:

8

One of the problem you have is:

  • "execution.command":"/bin/bash",
  • "execution.command.args":".-1"

Here you try to say that execution.command is a text. And execution.command.args is the args field inside execution.command object. Which would mean that execution.command should be both an object and a text field. Which is not possible.

You should rewrite your document field names in a way they are not seen as objects. Like:

{  
   "execution_command":"/bin/bash",
   "execution_command_args":".-1"
}
2 Likes
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.