Nested data not shown correctly with painless script

Hi all,

I may find issue with the combination of "nested data" and "painless script".

  • In case "painless script" is NOT used, documents are shown correctly.
  • In case "painless script" is used, they are shown wrongly. Of course, I never update documents in the script.

Please help to solve issue or analyze it further.


  • mapping
{
  "mappings": {
    "_doc": {
      "properties": {
        "study_id": {
          "type": "keyword"
        },
        "ext_number":{
          "type": "keyword"
        },
        "study_timestamp":{
          "type": "date"
        },
        "study_secondslong":{
          "type": "float"
        },
        "student_learn_rate":{
          "type": "float"
        },
        "additional_study_timestamp":{
          "type": "date"
        },
        "additional_data01":{
          "type": "keyword"
        },
        "additional_data02":{
          "type": "keyword"
        },
        "study_text":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "study_text_student":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "study_text_teacher":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "file_uri":{
          "type": "keyword"
        },
        "book_del_flg":{
          "type": "keyword"
        },
        "teach_cm":{
          "type": "double"
        },
        "student_cm":{
          "type": "double"
        },
        "claim_old":{
          "type": "integer"
        },
        "negative_task":{
          "type": "text"
        },
        "negative_score":{
          "type": "float"
        },
        "teacher":{
          "type": "keyword"
        },
        "teacher_flg":{
          "type": "keyword"
        },
        "claim":{
          "type": "integer"
        },
        "nested_data":{
          "type": "nested",
          "properties":{
            "learner_id":{
              "type": "keyword"
            },
            "version_number":{
              "type": "float"
            },
            "dict_name":{
              "type": "keyword"
            },
            "c_time":{
              "type": "float"
            },
            "u_time":{
              "type": "float"
            },
            "m_time1":{
              "type": "float"
            },
            "m_time2":{
              "type": "float"
            },
            "m_time3":{
              "type": "float"
            },
            "m_time4":{
              "type": "float"
            },
            "tf_done_flg":{
              "type": "keyword"
            },
           "cluster_done_flg":{
              "type": "keyword"
            },
            "evalexp_done_flg":{
              "type": "keyword"
            },
            "learner_data":{
              "type": "nested",
              "properties":{
                "study_id": {
                  "type": "keyword"
                },
                "turn":{
                  "type": "integer"
                },
                "learn_begin_timeoffset":{
                  "type": "float"
                },
                "learn_end_timeoffset":{
                  "type": "float"
                },
                "learn_confidence":{
                  "type": "keyword"
                },
                "words_confidence":{
                  "type": "text"
                },
                "get_words":{
                  "type": "text",
                  "fielddata": true
                },
                "present_rate":{
                  "type": "float"
                },
                "silent_time":{
                  "type": "float"
                },
                "learn_position":{
                  "type": "integer"
                },
                "study_reason_flg":{
                  "type": "keyword"
                },
                "_words_confidence":{
                  "type": "text"
                }
              }
            }
          }
        }
      }
    }
  }
}
  • only mutch_all query. NOP
GET school/_search
{
  "query": {
    "match_all": {}
  }
}
  • When returning "params._source" from painless script, the nested data is wrong for some documents.
GET school/_search
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "test": {
      "script": {
        "lang": "painless",
        "source": "
          return params._source
        "
      }
    }
  }
}
  • However, when returning "params._source.nested_data" from painless script, the nested data is correct for all documents.
GET school/_search
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "test": {
      "script": {
        "lang": "painless",
        "source": "
          return params._source.nested_data
        "
      }
    }
  }
}

None???

None??? again

Can you please come up with a fully fledged example include samples documents? It is really hard to understand or even interpret what you mean with wrong and correct, if you do not include responses and sample documents.

Thanks!

--Alex

2 Likes

Hi Alex,

Thank you for your response.
I found the issue with documents including Japanese language but have not reproduced it in English yet.
So I have not been able to post sample documents to this topic.

After I reproduce the issue in English, I will let you know.

By the way, is there any way (e.g. log with debug level) to analyze this issue further by myself?
As far as I looked for such way, I could't.

  • FYI

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.