Scripted fields: text field returns null

Elastic Stack Kibana
5

There are a couple of gotchas when working with text fields in painless.

Text fields are stored as a list of tokens in Lucene doc values, so this is what painless returns when you access them via doc. If that's what you want, then you'll need "fielddata": true for the text field. Without that setting, you will get an illegal_argument_exception when accessing a text field.

Typically, you'll want to use the keyword version of the field, as that will come back as the untokenized string.

However, if you have a long field and "ignore_above": 256, then the keyword field will not be available if the source data is longer than 256 characters.

Here's an example that demonstrates the difference between text and keyword fields.

PUT test
{
  "mappings": {
    "properties": {
      "test": {
        "type": "text",
        "fielddata": true,
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

PUT test/_doc/1
{
  "test": "abc 123 do re mi"
}

GET test/_search
{
  "runtime_mappings": {
    "mytext": {
      "type": "keyword",
      "script": {
        "source": "emit(doc['test'].value)"
      }
    },
    "mykeyword": {
      "type": "keyword",
      "script": {
        "source": "emit(doc['test.keyword'].value)"
      }
    }
  },
  "fields": [
    "mytext", "mykeyword"
  ]
}

Returns

    "hits" : [
      {
        "_index" : "test",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "test" : "abc 123 do re mi"
        },
        "fields" : {
          "mytext" : [
            "123"
          ],
          "mykeyword" : [
            "abc 123 do re mi"
          ]
        }
      }
    ]

I can't think of a case where doc would return null.

It would be easier to help with a minimal reproduction of the issue.