# Removing empty fields from an event (2024 edition!)

**URL:** <https://discuss.elastic.co/t/removing-empty-fields-from-an-event-2024-edition/352571>\
**Category:** Logstash\
**Created:** [February 5, 2024, 9:34pm UTC](https://discuss.elastic.co/t/removing-empty-fields-from-an-event-2024-edition/352571 "2024-02-05T21:34:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Supermathie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/supermathie/32/44936_2.png) [@Supermathie](https://discuss.elastic.co/u/Supermathie)\
**Post date:** [February 5, 2024, 9:34pm UTC](https://discuss.elastic.co/t/removing-empty-fields-from-an-event-2024-edition/352571/1 "2024-02-05T21:34:09Z")

</div>

Continuing the discussion from [Never ending story: how to check and remove empty fields, arrays etc](https://discuss.elastic.co/t/never-ending-story-how-to-check-and-remove-empty-fields-arrays-etc/154243):

There's no simple way to (recursively) get all field names in an event, but one can do this:

```plaintext
filter {
    ruby {
       init => "
       def remove_empty_fields(event)
         _find_empty_fields(event.to_hash) { |path| event.remove('[' + path.join('][') + ']') }
       end

       def _find_empty_fields(event, path = [], &blk)
         event.each do |k, v|
           curpath = path + [k]
           case v
           when nil
             yield curpath
           when ''
             yield curpath
           when []
             yield curpath
           when {}
             yield curpath
           when Hash
             _find_empty_fields(v, curpath, &blk)
           end
         end
       end
       "
       code => "remove_empty_fields(event)"
    }
}

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [February 5, 2024, 9:55pm UTC](https://discuss.elastic.co/t/removing-empty-fields-from-an-event-2024-edition/352571/2 "2024-02-05T21:55:15Z")

</div>

Another option is to use the Elastic approach with Elastic Agent integrations and remove those fields using a script processor.

If I'm not wrong all Elastic Agent integrations have this processor to remove empty/null fields.

```auto
  - script:
      description: Drops null/empty values recursively
      lang: painless
      source: |
        boolean drop(Object o) {
          if (o == null || o == "") {
            return true;
          } else if (o instanceof Map) {
            ((Map) o).values().removeIf(v -> drop(v));
            return (((Map) o).size() == 0);
          } else if (o instanceof List) {
            ((List) o).removeIf(v -> drop(v));
            return (((List) o).length == 0);
          }
          return false;
        }
        drop(ctx);

```

I created an ingest pipeline with only this processor and configured it to be the executed on all my data that is ingested using Logstash by setting an `index.final_pipeline` in the templates.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 4, 2024, 9:56pm UTC](https://discuss.elastic.co/t/removing-empty-fields-from-an-event-2024-edition/352571/3 "2024-03-04T21:56:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.