Nested issue

Elastic Stack Elasticsearch
1

Hi,

I have a little issue to use nested field in the mapping. I use elasticsearch 2.3.2

The mapping is the following:
"@stk": {
"properties": {
"@children": {
"type" : "nested",
"properties": {
"@name": {
"type": "string",
"index": "not_analyzed"
},
"@children": {
"type" : "nested",
"properties": {
"@name": {
"type": "string",
"index": "not_analyzed"
}
}....

Each children can have children up to 6 levels.

The data to insert is like :
"@stk":{"@name":"Root","@time":5,
"@children":[
{"@name":"child1","@time":5,
"@children":[
{"@name":"child1.1","@time":3},
{"@name":"child1.2","@time":2}
]
}
]}

The insertion error is "error"=>{"type"=>"illegal_argument_exception", "reason"=>"object mapping [@stk.@children] can't be changed from nested to non-nested"}

"type" : "object" works, but it's the nested type that I need for the queries.

Could you please kindly tell me what I do wrong ?

Thanks for your support,
Dominique

2

No one else has the same issue ?

3

This means that there is already a mapping for the children field of type
object. A field's type can't be changed after it is created. You should
create a new index wit you mapping that uses nested fields and reindex your
data from the current index into the new index.

4

I have deleted and created again the index, so it should not have any data inside. And I still have the issue. I even change the name of the "children" in case it interferes with the other index.

5

Can you share the rest calls that you executed?

6

Hi,
Thansk for you answer.
Below what I do.

curl -XDELETE 'http://myserver:9200/myindex'
curl -XPUT 'http://myserver:9200/_template/myindex' -d '
{
"template": "myindex",
"settings": {
"number_of_shards": 1,
"number_of_replicas": 2,
"index.refresh_interval": "5s"
},
"mappings": {
"logs":
{
"properties": {
"timestampGMT": {
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss.SSSSSS"
},
"@stk": {
"properties": {
"@name": {
"type": "string",
"index": "not_analyzed"
},
"@totalTime": { "type": "double"},
"@children": {
"type" : "nested",
"include_in_parent" : true,
"properties": {
"@name": {
"type": "string",
"index": "not_analyzed"
},
"@totalTime": { "type": "double"},
"@children": {
"type" : "nested",
"include_in_parent" : true,
"properties": {
"@name": {
"type": "string",
"index": "not_analyzed"
},
"@totalTime": { "type": "double"}
}
}
}
}
}
}
}
}
}
}'

I try to insert
{"@thr":0,"@customData":{"Data1":"1"},"@stk":{"@name":"Root","@totalTime":10,"@children":[{"@name":"SubRoot","@totalTime":10,"@children":[{"@name":"child1","@totalTime":5.5},{"@name":"child2","@totalTime":3.5},{"@name":"child3","@totalTime":1}]}]}}

And I get:
..."@stk"=>{"@name"=>"Root", "@totalTime"=>10, "@children"=>[{"@name"=>"SubRoot", "@totalTime"=>10, "@children"=>[{"@name"=>"child1", "@totalTime"=>#BigDecimal:2e0b1dd8,'0.55E1',2(4)}, {"@name"=>"child2", "@totalTime"=>#BigDecimal:724f9710,'0.35E1',2(4)}, {"@name"=>"child3", "@totalTime"=>1}]}]}}, "@stk"]}>>], :response=>{"create"=>{"_index"=>"myindex", "_type"=>"request1", "_id"=>"AVTHiW4YIkqJinos0jHU", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"object mapping [@stk.@children] can't be changed from nested to non-nested"}}}, :level=>:warn}

What do I perform wrong ?

Thanks.

7

I'm able to index the document using your reproduction. Are you sure that you are sending the index call to the right cluster?

These are the commands that I executed:

  • Delete index: curl -XDELETE "http://localhost:9200/myindex"
  • Put index template (I only changed settings.number_of_replicas to 0, so that I don't need two nodes when indexing) :
curl -XPUT "http://localhost:9200/_template/myindex" -d'
{
  "template": "myindex",
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "index.refresh_interval": "5s"
  },
  "mappings": {
    "logs": {
      "properties": {
        "timestampGMT": {
          "type": "date",
          "format": "yyyy/MM/dd HH:mm:ss.SSSSSS"
        },
        "@stk": {
          "properties": {
            "@name": {
              "type": "string",
              "index": "not_analyzed"
            },
            "@totalTime": {
              "type": "double"
            },
            "@children": {
              "type": "nested",
              "include_in_parent": true,
              "properties": {
                "@name": {
                  "type": "string",
                  "index": "not_analyzed"
                },
                "@totalTime": {
                  "type": "double"
                },
                "@children": {
                  "type": "nested",
                  "include_in_parent": true,
                  "properties": {
                    "@name": {
                      "type": "string",
                      "index": "not_analyzed"
                    },
                    "@totalTime": {
                      "type": "double"
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}'
  • Index document:
curl -XPUT "http://localhost:9200/myindex/logs/1" -d'
{
  "@thr": 0,
  "@customData": {
    "Data1": "1"
  },
  "@stk": {
    "@name": "Root",
    "@totalTime": 10,
    "@children": [
      {
        "@name": "SubRoot",
        "@totalTime": 10,
        "@children": [
          {
            "@name": "child1",
            "@totalTime": 5.5
          },
          {
            "@name": "child2",
            "@totalTime": 3.5
          },
          {
            "@name": "child3",
            "@totalTime": 1
          }
        ]
      }
    ]
  }
}'
8

Hi,

Thanks a lot. Indeed, with XPUT, I insert well in elasticsearch, so I guess my issue is in logstash. I'll check in this side.
I'll try to do a graph in kibana with split by @stk.@children.@children, and the result is absolutely false: child1, child2 and child3 have both 3.33 as totalTime instead of 5.5, 3.5 and 1.
Is there a way to do a visualization with right value per nested children ?

Thanks

9

Kibana doesn't support the nested query/aggregations, but there is an issue tracking this feature request: