Doc['message.keyword'] shows 0 value for given index but actually value is present in message field. What could be the reason?

[vishdubey (Vishal Dhar Dubey)]

below is the script field

GET log-2022.03.30/_search
{
  "script_fields": {
    "message": {
      "script": {
        "lang" : "painless",
        "source": "doc['message.keyword'].value"
      }
    }
  }
}

and here is the result I got

"failures" : [
      {
        "shard" : 0,
        "index" : "log-2022.03.30",
        "node" : "Jei35aKrR0-jxyKKld12jss",
        "reason" : {
          "type" : "script_exception",
          "reason" : "runtime error",
          "script_stack" : [
            "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:494)",
            "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:508)",
            "doc['message.keyword'].value",
            "                      ^---- HERE"
          ],
          "script" : "doc['message.keyword'].value",
          "lang" : "painless",
          "position" : {
            "offset" : 22,
            "start" : 0,
            "end" : 28
          },
          "caused_by" : {
            "type" : "illegal_state_exception",
            "reason" : "A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!"
          }
        }
      }
    ]

Kindly help on this.

[spinscale (Alexander Reelsen)]

Hey,

why is it not possible, that any of your documents does not have the message field set? Can you verify that using an exists query for that field, no documents are returned? Can you provide a fully reproducible example, that contains a sample document containing a message field and that error message, so we can reproduce locally?

--Alex

[vishdubey (Vishal Dhar Dubey)]

Hi Spinscale ,
every document have the message field with some data present to it.
I have verified it with query and got the data. but the only issue is when I try to read data with 'message.keyword' it gives no data. Below is the query :-

GET logstash-index-xxx/_search
{
  "size": 0, 
  "aggs": {
    "message": {
      "terms": {
        "script": {
          "source": "doc['message.keyword'].value"
        }
      }
    }
  }
}

As of now I have resolved the issue with enabling 'fielddata=true' but now the issue is when I try to see the logs for 7 days or more, It takes longer time and eventually fails giving internal server error ("failed to execute")