How to handle dot in field names in ES 7.0


(shashank) #1

I was recently hit by the fieldname-with-dots issue in ES. I search on google but found the answer for ES 2.X version while for other version none of the option work for me.

execution.command = /bin/bash
execution.command.args = emrStatusCluster

"Note : type_name is deprecated for ES 7.0 so default type is _doc "

I use curl -XPOST localhost:9200/index_name/_doc -H "Content-Type: application/json" -d @jsondata.json to load data and get following mssg on the console.

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't merge a non object mapping [execution.command] with an object mapping [execution.command]"}],"type":"illegal_argument_exception","reason":"Can't merge a non object mapping [execution.command] with an object mapping [execution.command]"},"status":400

I tried dot_expander but didnot find anything much useful and constructive.

Thanks in Advance ,
Shanky


(David Pilato) #2

What do you have in jsondata.json?


(shashank) #3

A single Json record in which execution.command and execution.command.argrs are present somewhere as key.


(David Pilato) #4

You need to fix its content probably.


(shashank) #5

I don't have any control on content since content comes from nifi where i call elastic search api is there any way to deal as it is given for ES 2.X version where we can allow dot in field name.

Thanks in advance.


(David Pilato) #6

I don't really know as I can't see the content.


(shashank) #7

Ok let me give you small view of Content which is in json format:

{  
   "s3.version":"mc5ETuM",
   "invokehttp.tx.id":"f75283d92c",
   "sqs.receipt.handle":"Zl/yWciVceGIiWiC5sNkQ=",
   "Server":"J4)",
   "s3.etag":"712a41",
   "listEMResponse":"",
   "indexed_file_path":"fasfd",
   "schemaName":"",
   "uuid":"cf",
   "sqs.message.id":"ae",
   "sqs.SenderId":"AI",
   "hash.value":"7b77",
   "tableName":"Table1",
   "execution.command":"/bin/bash",
   "filename":"1554448956907.csv",
   "s3.key":"/15544_TB_.csv",
   "execution.command.args":".-1"
}

Now the problem is execution.command.args and execution.command both contain data if i make execution.command type as object then on running it will give error what i did is :

PUT index_try
{
  "mappings": {
    "json": {
      "properties": {
        "created_at": {
          "type": "date",
          "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
        },
        "execution": {
          "type": "object",
          "properties": {
            "command": {
              "type": "object",
              "properties": {
                "args": {
                  "type": "text"
                }
              }
            }
          }
        }
      }
    }
  },
  "settings": {
    "number_of_shards": 6,
    "number_of_replicas": 1
  }
}

I refered following post may be this help you :slight_smile: