Failing to index a JSON object

Elastic Stack Elasticsearch
1

I have a file that contains JSON objects, one object per line. Following is an example of one (prettified) object. My intention is to do the following, in order to index the objects in ES:

cat foo.json | jq -c '. | {"index": {"_id": .sys_id, "_index": "test", "_type": "entry"}}, .' | curl -XPOST http://my.server.com:9200/_bulk --data-binary @-

Consider working.json which contains the following object:

{
"sdk_insights": {
"sun": [
  -1,
  -1,
  0,
  -1,
  -1,
  -1,
  0,
  1,
  -1,
  0,
  -1,
  0,
  -1
],
"age_value": "25-34"
},
"@timestamp": 1333375220000,
"sys_id": "9493-87c5"
}

Running the above command on working.json does the trick and the object is indexed in ES. However, if I have non-working.json which contains:

{
  "sdk_insights": {
    "sun": [
      -1,
      -1,
      0,
      -1,
      -1,
      -1,
      0,
      1,
      -1,
      0,
      -1,
      0,
      -1
    ],
    "wed": [
      0,
      0.5,
      0,
      0,
      -1,
      0,
      0,
      0.5,
      0,
      0,
      0
    ],
    "all_week": [
      0.26,
      0.094,
      0.11,
      0.15,
      0.25,
      0.18,
      0.19,
      0.23,
      0.11,
      0.19,
      0.2,
      0.26,
      0.16,
      0.13
    ],
    "age_value": "25-34"
  },
  "@timestamp": 1333375220000,
  "sys_id": "87c5-9493"
}

Then the magic is gone. I get the following error: {"took":18,"errors":true,"items":[{"index":{"_index":"test","_type":"entry","_id":"87c5-9493","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"mapper [sdk_insights.wed] of different type, current_type [long], merged_type [double]"}}}}]}

I understand that I have to fix something with the mapping, but I failed to do it. I would appreciate any kind of help. Thanks!

2

The sdk_insights.web field has been mapped as a long but you're trying to stuff floating-point values into the field. You'll have to delete the current index and create a new one with the correct mapping for your fields.

3

So the hack is to create the index with the following mapping:

PUT /test
{
  "mappings": {
    "entry": {
      "properties": {
        "sdk_insights": {
          "type": "nested", 
          "properties": {
            "wed":    { "type": "double"  },
            "all_week": { "type": "double"  }
          }
        }
      }
    }
  }
}

Is that correct?

Source: https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-mapping.html

4

Why would you call that a hack? But yes, it looks correct.