# Failed to start filebeat\_ drop\_fields processors

**URL:** <https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154>\
**Category:** Beats\
**Tags:** beats-module, filebeat\
**Created:** [March 9, 2022, 12:55am UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154 "2022-03-09T00:55:56Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\
**Post date:** [March 9, 2022, 12:55am UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/1 "2022-03-09T00:55:56Z")

</div>

good day

I have a problem when I start filebeat I get the following error:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/5/85e2c93b325c9d446029d6ec3fe39613bd70dbfe.png)

and this is my configuration in filebeat.yml

```auto
# ================================= Processors =================================
processors:
# - add_host_metadata:
 # when.not.contains.tags: forwarded
 # - add_cloud_metadata: ~
 # - add_docker_metadata: ~
 # - add_kubernetes_metadata: ~
 - drop_fields:
      when:
       network:
        observer.ip: '10.252.132.138'
      fields: ["agent.name", "agent.hostname", "agent.type", "destination.locality"]
      ignore_missing: true

```

I have a processor to drop certain netflow fields that it sends me, since it is too much information that it sends me to elastic and I only need it to send me specific fields to the condition that I apply to an IP observer, that is to say that it sends me data of only that IP.

for that reason I kindly ask for your help to solve this case because it is important, and this can be useful to more people who have the same problem, when I want to drop fields and send data from a specific ip and also send only the fields I need.

thank you I hope your prompt response with this. ☹

---

<div class="post-metadata">

**Author:** ![Tetiana\_Kravchenko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tetiana_kravchenko/32/102683_2.png) [@Tetiana\_Kravchenko](https://discuss.elastic.co/u/Tetiana_Kravchenko)\
**Post date:** [March 9, 2022, 2:17pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/2 "2022-03-09T14:17:37Z")

</div>

Hi @Juan_David_Jaramillo !  
Are you sure that the problem is actually with `drop_fields` processor? Does filebeat start, when this processor is commented out? Could you please provide filebeat logs?

---

<div class="post-metadata">

**Author:** ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\
**Post date:** [March 9, 2022, 2:56pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/3 "2022-03-09T14:56:28Z")

</div>

that's right, I have commented the line of code of "drop\_fields" and it works correctly, but when I want to run it with the "drop\_fields" active it doesn't execute filebeat and I get the previous error

---

<div class="post-metadata">

**Author:** ![Tetiana\_Kravchenko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tetiana_kravchenko/32/102683_2.png) [@Tetiana\_Kravchenko](https://discuss.elastic.co/u/Tetiana_Kravchenko)\
**Post date:** [March 14, 2022, 1:22pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/4 "2022-03-14T13:22:01Z")

</div>

Hi @Juan_David_Jaramillo !

The error:

```auto
ERROR	instance/beat.go:1015	Exiting: Failed to start crawler: starting input failed: Error while initializing input: invalid CIDR address: 10.252.132.138
failed to parse CIDR, values must be an IP address and prefix length, like '192.0.2.0/24' or '2001:db8::/32', as defined in RFC 4632 and RFC 4291.

```

you should use:

```auto
when:
  network:
    observer.ip: '10.252.132.138/32'

```

---

<div class="post-metadata">

**Author:** ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\
**Post date:** [March 14, 2022, 3:38pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/5 "2022-03-14T15:38:37Z")

</div>

thank you very much for your answer, I have a question, is there a way to get the fields that I only need, since dropping fields is more tedious because of the amount that comes out of netflow, and the ones I need to send are few that I already have mapped, but my question is if there is any filter to specify which fields to send to Elasticsearch? try with the filter "prune - whitelist" but it did not work for my case ☹

---

<div class="post-metadata">

**Author:** ![Tetiana\_Kravchenko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tetiana_kravchenko/32/102683_2.png) [@Tetiana\_Kravchenko](https://discuss.elastic.co/u/Tetiana_Kravchenko)\
**Post date:** [March 14, 2022, 6:13pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/6 "2022-03-14T18:13:04Z")

</div>

did you try [`include_fields`](https://www.elastic.co/guide/en/beats/filebeat/current/include-fields.html) processor?

---

<div class="post-metadata">

**Author:** ![Juan\_David\_Jaramillo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_david_jaramillo/32/76831_2.png) [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\
**Post date:** [March 14, 2022, 6:53pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/7 "2022-03-14T18:53:03Z")

</div>

thank you very much that was just what I needed!

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/1/f1a7e055ce47c52d5ef0e56e0e976289e1f53eab.png)

```auto
input {
  beats {
    port => 5044
    add_field => { "Tipo" => "netflow-test" }
  }
}

#filter {
#if [observer][ip] == "10.20.248.34" {
# mutate {
# remove_field => [[source][locality],[source][packet]]
#}}}

#filter {
 # prune {
  # interpolate => true
   # whitelist_names => ["[source][ip]", "[observer][ip]$" ]
   # }
#}

#filter {
 # mutate {
  # remove_field => ["%{@version}","%{@type}","%{agent}","%{netflow}","%{network}","%{fields}"]
  # }
#}
#filter {
 # mutate {
  # remove_field => ["[agent][id]", "[agent][hostname]", "[agent][name]", "[agent][type]", "[agent][version]"]
# remove_field => ["[cloud][account][id]", "[cloud][availability_zone]", "[cloud][instance][id]", "[cloud][instance][name]", "[cloud][machine][type]", "[cloud][project][id]", "[cloud][provider]", "[cloud][servi>
# remove_field => ["[destination][locality]", "[destination][port]", "[ecs][version]", "[event][action]", "[event][category]", "[event][created]" ,"[event][dataset]", "[event][duration]", "[event][end]", "[even>
# remove_field => ["[fileset][name]", "[input][type]", "[netflow][exporter][address]", "[netflow][exporter][source_id]", "[netflow][exporter][timestamp]", "[netflow][exporter][uptime_millis]", "[netflow][export>
# remove_field => ["[agent][ephemeral_id]", "[event][created]", "[event][end]", "[event][start]", "[flow][id]", "[flow][locality]", "[related][ip]", "[service][type]", "[source][bytes]", "[source][locality]", ">
# remove_field => ["[netflow][bgp_destination_as_number]", "[netflow][bgp_next_hop_ipv4_address]", "[netflow][bgp_source_as_number]", "[netflow][destination_ipv4_address]", "[netflow][destination_ipv4_prefix_l>
 # }
#}
#filter {
#mutate { add_field => { "[@metadata][source]" => "%{[source][ip]}" "[@metadata][observer]" => "%{[observer][ip]}" } }
 # prune {
 # whitelist_names => ["@timestamp", "host"]
 # add_field => { "[source][ip]" => "%{[@metadata][source]}" "[observer][ip]" => "%{[@metadata][observer]}" }
 #
#}
#}

```

before I had made a filter to delete fields from logstash because the "drop\_fields" didn't work, but apparently netflow has too many fields that I don't need and it also generates new fields, so it was very difficult to consider.

thank you very much for your help!

See you soon!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 11, 2022, 8:53pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-drop-fields-processors/299154/8 "2022-04-11T20:53:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.