Default values for missing fields

Elastic Stack Elasticsearch
1

Can I create a field in a mapping that will be indexed with some default value if that field is not present in the document being indexed?

In other words, if I have a mapping with the field {"Rating": "integer"}, and then I index a document that does not include this field, can I have the field "Rating" be indexed, for example, with a value of zero?

2

Take a look at null_value | Elasticsearch Guide [8.1] | Elastic

3

I don't think null_value does what I want (unless I don't understand the documentation). Here is precisely what I want to do. I want to create an index with the following mapping:

PUT /testindex
{
  "mappings": {
    "properties": {
      "email":  { "type": "keyword"}, 
      "name":   { "type": "text"},     
      "age":    { "type": "integer"}  
    }
  }
}

Then I want to index the following document:

PUT /testindex/_doc/1
{
  "name":"John Smith",
  "email":"jsmith@email.com"
}

[note that I only indexed two of the fields defined in my mapping]
When I then retrieve that document like this:

GET /testindex/_doc/1

I am HOPING to get all three of my fields, but with the missing field ("age") set to some default value. But I only get the following response:

{
  "_index" : "testindex",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "John Smith",
    "email" : "jsmith@email.com"
  }
}

My reading of the null_value parameter seems to indicate that it only applies when I explicitly make one of the fields null.

Is there any way for me to force "age" to be set to a default value?

4

There's no other way to do that in Elasticsearch unfortunately.

5

I think that the only way to do something like that is using an ingest pipeline with some processors that would add the field if it does not exist.

It is not the cleanest way as it depends on the number of fields you want to add, but it works. You could create an ingest pipeline, configure your template to use this pipeline as the index.final_pipeline and then add some set processors with conditionals to check if the field exists.

An example of a set processor would be this:

    {
      "set": {
        "field": "age",
        "value": 0,
        "override": false,
        "if": "ctx.age == null",
        "ignore_failure": true
      }
    }
1 Like
6

That's a great solution!

7

Thanks everyone, I appreciate the help.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.