# Nested data not shown correctly with painless script

**URL:** <https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012>\
**Category:** Elasticsearch\
**Created:** [November 16, 2018, 9:15am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012 "2018-11-16T09:15:29Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![junkoshimane](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/junkoshimane/32/36594_2.png) [@junkoshimane](https://discuss.elastic.co/u/junkoshimane)\
**Post date:** [November 16, 2018, 9:15am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012/1 "2018-11-16T09:15:29Z")

</div>

Hi all,

I may find issue with the combination of "nested data" and "painless script".

- In case "painless script" is NOT used, documents are shown correctly.
- In case "painless script" is used, they are shown wrongly. Of course, I never update documents in the script.

Please help to solve issue or analyze it further.

* * *

- mapping

```auto
{
  "mappings": {
    "_doc": {
      "properties": {
        "study_id": {
          "type": "keyword"
        },
        "ext_number":{
          "type": "keyword"
        },
        "study_timestamp":{
          "type": "date"
        },
        "study_secondslong":{
          "type": "float"
        },
        "student_learn_rate":{
          "type": "float"
        },
        "additional_study_timestamp":{
          "type": "date"
        },
        "additional_data01":{
          "type": "keyword"
        },
        "additional_data02":{
          "type": "keyword"
        },
        "study_text":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "study_text_student":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "study_text_teacher":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "file_uri":{
          "type": "keyword"
        },
        "book_del_flg":{
          "type": "keyword"
        },
        "teach_cm":{
          "type": "double"
        },
        "student_cm":{
          "type": "double"
        },
        "claim_old":{
          "type": "integer"
        },
        "negative_task":{
          "type": "text"
        },
        "negative_score":{
          "type": "float"
        },
        "teacher":{
          "type": "keyword"
        },
        "teacher_flg":{
          "type": "keyword"
        },
        "claim":{
          "type": "integer"
        },
        "nested_data":{
          "type": "nested",
          "properties":{
            "learner_id":{
              "type": "keyword"
            },
            "version_number":{
              "type": "float"
            },
            "dict_name":{
              "type": "keyword"
            },
            "c_time":{
              "type": "float"
            },
            "u_time":{
              "type": "float"
            },
            "m_time1":{
              "type": "float"
            },
            "m_time2":{
              "type": "float"
            },
            "m_time3":{
              "type": "float"
            },
            "m_time4":{
              "type": "float"
            },
            "tf_done_flg":{
              "type": "keyword"
            },
           "cluster_done_flg":{
              "type": "keyword"
            },
            "evalexp_done_flg":{
              "type": "keyword"
            },
            "learner_data":{
              "type": "nested",
              "properties":{
                "study_id": {
                  "type": "keyword"
                },
                "turn":{
                  "type": "integer"
                },
                "learn_begin_timeoffset":{
                  "type": "float"
                },
                "learn_end_timeoffset":{
                  "type": "float"
                },
                "learn_confidence":{
                  "type": "keyword"
                },
                "words_confidence":{
                  "type": "text"
                },
                "get_words":{
                  "type": "text",
                  "fielddata": true
                },
                "present_rate":{
                  "type": "float"
                },
                "silent_time":{
                  "type": "float"
                },
                "learn_position":{
                  "type": "integer"
                },
                "study_reason_flg":{
                  "type": "keyword"
                },
                "_words_confidence":{
                  "type": "text"
                }
              }
            }
          }
        }
      }
    }
  }
}

```

- only mutch\_all query. NOP

```auto
GET school/_search
{
  "query": {
    "match_all": {}
  }
}

```

- When returning "params.\_source" from painless script, the nested data is wrong for some documents.

```auto
GET school/_search
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "test": {
      "script": {
        "lang": "painless",
        "source": "
          return params._source
        "
      }
    }
  }
}

```

- However, when returning "params.\_source.nested\_data" from painless script, the nested data is correct for all documents.

```auto
GET school/_search
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "test": {
      "script": {
        "lang": "painless",
        "source": "
          return params._source.nested_data
        "
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![junkoshimane](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/junkoshimane/32/36594_2.png) [@junkoshimane](https://discuss.elastic.co/u/junkoshimane)\
**Post date:** [November 21, 2018, 1:26am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012/2 "2018-11-21T01:26:11Z")

</div>

None???

---

<div class="post-metadata">

**Author:** ![junkoshimane](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/junkoshimane/32/36594_2.png) [@junkoshimane](https://discuss.elastic.co/u/junkoshimane)\
**Post date:** [December 3, 2018, 8:40am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012/3 "2018-12-03T08:40:35Z")

</div>

None??? again

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [December 3, 2018, 9:51am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012/4 "2018-12-03T09:51:58Z")

</div>

Can you please come up with a fully fledged example include samples documents? It is really hard to understand or even interpret what you mean with wrong and correct, if you do not include responses and sample documents.

Thanks!

--Alex

---

<div class="post-metadata">

**Author:** ![junkoshimane](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/junkoshimane/32/36594_2.png) [@junkoshimane](https://discuss.elastic.co/u/junkoshimane)\
**Post date:** [December 4, 2018, 6:09am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012/5 "2018-12-04T06:09:59Z")

</div>

Hi Alex,

Thank you for your response.  
I found the issue with documents including Japanese language but have not reproduced it in English yet.  
So I have not been able to post sample documents to this topic.

After I reproduce the issue in English, I will let you know.

By the way, is there any way (e.g. log with debug level) to analyze this issue further by myself?  
As far as I looked for such way, I could't.

---

<div class="post-metadata">

**Author:** ![junkoshimane](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/junkoshimane/32/36594_2.png) [@junkoshimane](https://discuss.elastic.co/u/junkoshimane)\
**Post date:** [December 21, 2018, 6:22am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012/6 "2018-12-21T06:22:14Z")

</div>

- FYI

> <https://stackoverflow.com/questions/42009778/elasticsearch-painless-script-how-to-iterate-in-an-array-of-nested-objects>

> [@Painless: problem accessing nested object using dot notation](https://discuss.elastic.co/t/painless-problem-accessing-nested-object-using-dot-notation/150599):
>
> Hi, I'm having problems reindexing data using a painless script. The nested field context.data.result is mapped as an object in the destination index, but in some cases the source data is either a string or an array of longs The script attempts to: Check if the field exists Check the field's type If it is a string or array create a temporary string / array variable assign the value of the offending field to this variable change the value of the field to be a new object (passed in via param…

> [@Painless access to nested objects](https://discuss.elastic.co/t/painless-access-to-nested-objects/73882/):
>
> Hi, I'm facing the following situation : My data has a structure similar to this { "site1" : { "events": [{"event1", "date1"}, {"event1","date1-2"},{"event2","date3"}}] ... }, "site2" : { "events": [{"event2", "date4"}, {"event1","date5"},{"event3","date6}}] ... }, ... } What I want is : Count the number of sites where a specific event occurs during a given time range. Count for each site the numer of occurences during a specific time range. I could get the first via a simple …

> [@Get nested fields data with Painless scripting](https://discuss.elastic.co/t/get-nested-fields-data-with-painless-scripting/75851):
>
> I'm trying to access nested fields data through Painless inline scripting and can't seem to be able to get it to work. Without getting into too many details, the reason for this is that we have certain fields that support a default value, and an optional custom override. To avoid having to return both fields and then have the application decide which one to use, I'd like to use a scripted field to check if the override has value and return that, or otherwise fallback to the default value. I've …

> [@Use normal and nested fields in same Painless script query?](https://discuss.elastic.co/t/use-normal-and-nested-fields-in-same-painless-script-query/85470):
>
> I have a mapping like this: { printings: { type: "nested", properties: { prop1: {type: "number"} } }, prop2: {type: "number"} } I then want to build a Painless query like this: "script": { "lang": "painless", "inline": "doc['prop1'] \> (3 \* doc['printings.prop2'])" } However testing this in Sense doesn't work. If I replace the nested prop2 with a simple number then it does work. Is there a way to access both root and nested props in a sin…

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 18, 2019, 6:22am UTC](https://discuss.elastic.co/t/nested-data-not-shown-correctly-with-painless-script/157012/7 "2019-01-18T06:22:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.