Enabling null values on expanded document in Kibana Discover

[PodarcisMuralis (Murat K.)]

Hi.

I am using version 8.5.3.

• I created a Data View under Stack Management.
• Displayed an index with multiple fields successfully
• If we filter results and click the diagonal button on document, expanded document comes in from the rights as slide in.
• It displays the document in detail.

But when my custom_field has value "null" it does not display my field custom_field even in json section.

Is there any way to display custom_field?

Thanks.

[jsanz (Jorge Sanz)]

The noSQL nature of the Elasticsearch database involves dynamic mapping, so a document with a field with a null value is not indexed.

From the null_value property documentation:

A null value cannot be indexed or searched. When a field is set to null, (or an empty array or an array of null values) it is treated as though that field has no values.

null_value | Elasticsearch Guide [8.9] | Elastic

You may want to leverage that parameter to explicitly mark null values but not all field types support it.

Here some testing to play with:

# Clean up
DELETE discuss-341052

# Create an index with some null values
PUT discuss-341052
{
  "settings": {
    "number_of_replicas": 0
  },
  "mappings": {
    "properties": {
      "title": { "type": "text"},
      "category": { "type": "keyword", "null_value": "NULL"},
      "description": { "type": "text"},
      "score": { "type": "float", "null_value": -1}
    }
  }
}

# Add some data with some rows without fields, 
# others with explicit null values
POST discuss-341052/_bulk
{ "index": {} }
{ "title": "a", "category": "a", "description": "a desc", "score": 1}
{ "index": {} }
{ "title": "b", "category": "a", "description": "a desc", "score": 1}
{ "index": {} }
{ "title": "c", "category": "b", "description": "a desc", "score": 1}
{ "index": {} }
{ "title": "c", "category": "b"}
{ "index": {} }
{ "title": "c", "category": "c", "description": null, "score": -1}
{ "index": {} }
{ "title": "d", "category": "NULL", "description": "a desc", "score": -1}

# Create a Kibana Data View
POST kbn:/api/data_views/data_view
{
  "data_view": {
    "title": "discuss-341052"
  }
}

So on Discover some documents don't have the missing fields as you mentioned

But the last document has the explicit (and searchable) null documents