# Prune filter does not work

**URL:** https://discuss.elastic.co/t/prune-filter-does-not-work/299274
**Category:** Logstash
**Created:** [March 10, 2022, 1:24am UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274 "2022-03-10T01:24:58Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)
#### Post date: [March 10, 2022, 1:24am UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/1 "2022-03-10T01:24:59Z")

</div>

good morning, hope you are well

I have a problem with the "prune" filter in logstash and I have a configuration file in logstash in which the objective is to take the data from filebeat-netflow and go through the pipe in logstash and from there I filter the fields that I need to send to Elasticsearch, I have this configuration, but I do not know if the fields are well mapped according to the syntax that handles netflow for those fields, but the filter does not work, that is to say it does nothing and when running the script is stuck in listening and does not send data.

I need your help with this please, and so feedback about the information of the filter 'prune' with the function 'whitelist\_names', as there is no information or consistent examples in the documentation in elastic, do not know how to map correctly filebear-netflow fields and that filter is required as the netflow packs many unnecessary fields that consume significant storage in my elastic cloud, thanks,

```auto
input {
  beats {
    port => 5044
  }
}

filter {
    prune {
        interpolate => true
        whitelist_names => ["[source][ip]", "[observer][ip]" ]
    }
}

output {
if [observer][ip] == "10.20.248.34" {
  #stdout{ }
  elasticsearch {
    hosts => ["https://xxxxxxxxxxxxx.us-central1.gcp.cloud.es.io:9243"]
    user => "elastic"
    password => "xxxxxxxx"
    index => "ntw"
    }
 }
}

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 10, 2022, 2:53am UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/2 "2022-03-10T02:53:00Z")

</div>

The documentation for the prune filter notes that

> This filter currently only support operations on top-level fields, i.e. whitelisting and blacklisting of subfields based on name or value does not work.

See also [this](https://discuss.elastic.co/t/ruby-filter-to-whitelist-nested-fields/298424/2) post.

---

<div class="post-metadata">

### Author: ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)
#### Post date: [March 10, 2022, 6:31pm UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/3 "2022-03-10T18:31:49Z")

</div>

Ok, thanks for your answer, but then what filter can I apply to choose which fields I want to send only? so it is not a whitelist, any other script that I can use to indicate and specify to send me only the fields I need?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 10, 2022, 7:03pm UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/4 "2022-03-10T19:03:44Z")

</div>

For your use-case you could try something like this to save the fields you want, and then delete everything

```
    mutate { add_field => { "[@metadata][source]" => "%{[source][ip]}" "[@metadata][observer]" => "%{[observer][ip]}" } }
    prune {
        whitelist_names => ["[A][field][that][does][not][exist]" ]
        add_field => { "[source][ip]" => "%{[@metadata][source]}" "[observer][ip]" => "%{[@metadata][observer]}" }
    }

```

However, I think it more likely you will want

```
        whitelist_names => ["@timestamp", "host"]
```

---

<div class="post-metadata">

### Author: ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)
#### Post date: [March 10, 2022, 7:41pm UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/5 "2022-03-10T19:41:21Z")

</div>

```auto
filter {
mutate { add_field => { "[@metadata][source]" => "%{[source][ip]}" "[@metadata][observer]" => "%{[observer][ip]}" } }
    prune {
        whitelist_names => ["@timestamp", "host"]
        add_field => { "[source][ip]" => "%{[@metadata][source]}" "[observer][ip]" => "%{[@metadata][observer]}" }
    }
}

```

I tried it but it doesn't work, I run the logstash file and it looks like this:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/1/511c08d584f81536df129f96b787eddb10dbfd83.png)

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 10, 2022, 8:03pm UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/6 "2022-03-10T20:03:44Z")

</div>

It looks like it is waiting for new events to arrive.

---

<div class="post-metadata">

### Author: ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)
#### Post date: [March 10, 2022, 8:15pm UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/7 "2022-03-10T20:15:00Z")

</div>

Yes, but it stays there and does not generate any event and I have waited a long time, because before it did send at once. ☹

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 7, 2022, 8:15pm UTC](https://discuss.elastic.co/t/prune-filter-does-not-work/299274/8 "2022-04-07T20:15:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
