Nested data not shown correctly with painless script


(Junkoshimane) #1

Hi all,

I may find issue with the combination of "nested data" and "painless script".

  • In case "painless script" is NOT used, documents are shown correctly.
  • In case "painless script" is used, they are shown wrongly. Of course, I never update documents in the script.

Please help to solve issue or analyze it further.


  • mapping
{
  "mappings": {
    "_doc": {
      "properties": {
        "study_id": {
          "type": "keyword"
        },
        "ext_number":{
          "type": "keyword"
        },
        "study_timestamp":{
          "type": "date"
        },
        "study_secondslong":{
          "type": "float"
        },
        "student_learn_rate":{
          "type": "float"
        },
        "additional_study_timestamp":{
          "type": "date"
        },
        "additional_data01":{
          "type": "keyword"
        },
        "additional_data02":{
          "type": "keyword"
        },
        "study_text":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "study_text_student":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "study_text_teacher":{
          "type": "text",
          "analyzer" : "kuromoji"
        },
        "file_uri":{
          "type": "keyword"
        },
        "book_del_flg":{
          "type": "keyword"
        },
        "teach_cm":{
          "type": "double"
        },
        "student_cm":{
          "type": "double"
        },
        "claim_old":{
          "type": "integer"
        },
        "negative_task":{
          "type": "text"
        },
        "negative_score":{
          "type": "float"
        },
        "teacher":{
          "type": "keyword"
        },
        "teacher_flg":{
          "type": "keyword"
        },
        "claim":{
          "type": "integer"
        },
        "nested_data":{
          "type": "nested",
          "properties":{
            "learner_id":{
              "type": "keyword"
            },
            "version_number":{
              "type": "float"
            },
            "dict_name":{
              "type": "keyword"
            },
            "c_time":{
              "type": "float"
            },
            "u_time":{
              "type": "float"
            },
            "m_time1":{
              "type": "float"
            },
            "m_time2":{
              "type": "float"
            },
            "m_time3":{
              "type": "float"
            },
            "m_time4":{
              "type": "float"
            },
            "tf_done_flg":{
              "type": "keyword"
            },
           "cluster_done_flg":{
              "type": "keyword"
            },
            "evalexp_done_flg":{
              "type": "keyword"
            },
            "learner_data":{
              "type": "nested",
              "properties":{
                "study_id": {
                  "type": "keyword"
                },
                "turn":{
                  "type": "integer"
                },
                "learn_begin_timeoffset":{
                  "type": "float"
                },
                "learn_end_timeoffset":{
                  "type": "float"
                },
                "learn_confidence":{
                  "type": "keyword"
                },
                "words_confidence":{
                  "type": "text"
                },
                "get_words":{
                  "type": "text",
                  "fielddata": true
                },
                "present_rate":{
                  "type": "float"
                },
                "silent_time":{
                  "type": "float"
                },
                "learn_position":{
                  "type": "integer"
                },
                "study_reason_flg":{
                  "type": "keyword"
                },
                "_words_confidence":{
                  "type": "text"
                }
              }
            }
          }
        }
      }
    }
  }
}
  • only mutch_all query. NOP
GET school/_search
{
  "query": {
    "match_all": {}
  }
}
  • When returning "params._source" from painless script, the nested data is wrong for some documents.
GET school/_search
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "test": {
      "script": {
        "lang": "painless",
        "source": "
          return params._source
        "
      }
    }
  }
}
  • However, when returning "params._source.nested_data" from painless script, the nested data is correct for all documents.
GET school/_search
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "test": {
      "script": {
        "lang": "painless",
        "source": "
          return params._source.nested_data
        "
      }
    }
  }
}

Painlessスクリプト利用時の想定外の挙動
(Junkoshimane) #2

None???


(Junkoshimane) #3

None??? again


(Alexander Reelsen) #4

Can you please come up with a fully fledged example include samples documents? It is really hard to understand or even interpret what you mean with wrong and correct, if you do not include responses and sample documents.

Thanks!

--Alex


(Junkoshimane) #5

Hi Alex,

Thank you for your response.
I found the issue with documents including Japanese language but have not reproduced it in English yet.
So I have not been able to post sample documents to this topic.

After I reproduce the issue in English, I will let you know.

By the way, is there any way (e.g. log with debug level) to analyze this issue further by myself?
As far as I looked for such way, I could't.