How to count empty fields zero

Elastic Stack Logstash
1

I have a dataset like this:

image
image588×806 12 KB

As shown, there are some empty fields on the dataset. While indexing that file onto ES, I want to fill these empty fields with "0" automatically. How could I do that on Logstash?

2

To be more concise, I want to pass all blank cells as "0" to ES index

3

Selam Umut,
You can use an ingest pipeline and add that ingest pipeline into index settings. In that way, the fields with null value will have 0 value.

Here is an example for you:

PUT _ingest/pipeline/convert_empty_to_zero
{
  "processors": [
    {
      "script": {
        "source": """
          for (entry in ctx.entrySet()) {
            if (entry.getValue() == '') {
              ctx[entry.getKey()] = 0;
            }
          }
        """
      }
    }
  ]
}

PUT coronavirus-1/_settings
{
  "index.default_pipeline": "convert_empty_to_zero"
}

PUT coronavirus-1/_doc/1
{
  "country": "Turkiye",
  "cumulative": 100,
  "daily_new": 10,
  "active_cases": ""
}

image
image1920×797 78.6 KB

Note: I recommend to use the ingest pipeline but if you wish you can also do that on Logstash side. Check this discussion for how.
Note2: You can also update your index mapping with null_value | Elasticsearch Guide [8.15] | Elastic but it looks like it's not fit for your use case.

2 Likes