Update documents with painless - nested and not nested objects with the same name

[Martin_Becker]

I messed up my documents and want to clean them up. I want to do this using _update_by_query and painless script.

I run Elastic version 5.6 and my documents hold nested objects which hold tracking info as a nested object like this:

"tracking": {
  "triggered": false,
  "member": false
},

Due to wrong use of the Java API I ended up with documents holding next to the nested object some additional variables as follows:

"tracking": {
  "triggered": false,
  "member": false
},
"tracking.member": true,
"tracking.triggered": true

The problem now is that Kibana uses both of these fields. However we filter the data (true or false) we always get an answer because there are both variants. It seems Kibana does not differentiate between the nested and the non-nested variables.

I want to get rid of the variables of the not-nested format. But I do not know how to identify the documents.

Here is my update script which updates the nested objects:

POST my-tracking_index/_update_by_query?conflicts=proceed
{
  "query": {
    "match": {
      "_id": "17231723681231923"
    }
  },
        "script": {
          "source": """
ctx._source.tracking.member = false;
ctx._source.tracking.triggered = false;
"""
        }
}

But I want to remove the not-nested objects (or set their value null). How do I do this?

Thank you.

[Martin_Becker]

I found the solution for this problem.

// removing the unwanted not-nested fields
if (ctx._source['tracking.member'] != null) {
    ctx._source.remove("tracking.member");
}

// editing the nested fields
ctx._source.tracking.member = true;

So basically one needs to put the non-nested fields in brackets .

I still wonder how the update mechanism of the Java API messed it up and will investigate further.