Partial rows buggy Kibana 5.6.8

[ErikvdVen (Erik Van De Ven)]

I would like to create a table visualisation for a dashboard, which need to view all documents even when some of the columns are missing. So I tried to enable the option "Show partial rows", which does not seem to work properly. The "show partial rows" options still doesn't display some documents, even if there is no reason for not showing them, I know this because I show the raw results as a separate visualisation on the same dashboard.

Now I am looking for an alternative. I tried to update the corresponding fields, within my template (by adding a null_value) and reindex all documents:

PUT _template/documents*
{
    "template": "my_template"
    "mappings": {
        "_default_": {
           "properties" {
              "MyField": {
                 "type": "text",
                 "fields": {
                     "keyword": {
                        "type":  "keyword",
                        "null_value" : "",
                     }
                 }
             }
          }
       }
    }
}

Unfortunately this does not work, somehow it still doesn't add "MyField" with a default value, to all documents. Guess the "null_value" only allows me to replace an empty value with a default value, if the document actually does contain that specific key.

Anyone who has another solution for this problem? The only solution I can come up with is actually add the keys to the documents with a default value using Logstash, at the moment of indexing. But it would be nicer if there is a simpler solution for this.

[Brandon_Kobel]

Hey @ErikvdVen, the null_value parameter only affects the index/searching of the document, it doesn't actually change what is reflected in the "_source" itself.

Have you tried using a Scripted Field in Kibana?

[ErikvdVen (Erik Van De Ven)]

Thanks for your response @Brandon_Kobel! I've taken a look at Scripted Fields, as you suggested. It certainly looks like a very promising solution! I'm able to run some tests the day after tomorrow, I will let you know the outcome

[ErikvdVen (Erik Van De Ven)]

Thanks @Brandon_Kobel for your help! Scripted fields work very well! The table visualisations return much better results than when I used the "Show partial rows" option.

For whoever is interested, this is the code I use for each column in my table:

if(!doc.containsKey('MyField.keyword') || doc['MyField.keyword'].empty || doc['MyField.keyword'] == null)
{
    return "<empty>";
}else{
    return doc['MyField.keyword'].value;
}

I still have to make sure the size of terms is big enough, so I will receive all results instead of just a part of it... Guess this is still an issue: https://github.com/elastic/kibana/issues/9677