# How to drop events only from specific module

**URL:** <https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 3, 2020, 4:15pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175 "2020-11-03T16:15:55Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [November 3, 2020, 4:15pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/1 "2020-11-03T16:15:55Z")

</div>

Hello,

I want to drop all events that filebeat was unable to parse.

I tried a few things my last attempt looks like this.

> ```
> processors:
> - drop_event:
> when:
> and:
> - equals.event.module: cisco
> - equals.message: "failed to find message"
> 
> ```

But I can still see the events with the message "failed to find message" in kibana log stream.  
How can I make it work?

Filebeat version is 7.9.2

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [November 4, 2020, 3:11pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/2 "2020-11-04T15:11:16Z")

</div>

Is that yaml correctly indented? Can you try playing with the indentations of the yamls? It's a common source of issues

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [November 4, 2020, 3:19pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/3 "2020-11-04T15:19:16Z")

</div>

The config is loading and starting. When I had a tab in ymal there was an issue with the starting of filebeat.

the weird thing is when I do something like this

```
POST filebeat-7.9.2-cisco-2020.11.03/_search
{
   "size": 1,
   "sort": { "@timestamp": "desc"},
   "query": {
      "match": {"message":"failed to find message"}
   }
}

```

This is a document I get back.

```
{
  "took" : 1202,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 8374,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "filebeat-7.9.2-cisco-2020.11.03",
        "_type" : "_doc",
        "_id" : "EYtVkHUBMI-dbCtF8X18",
        "_score" : null,
        "_source" : {
          "agent" : {
            "hostname" : "name",
            "name" : "name",
            "id" : "4b069842-f4a1-4420-9a89-7e093d2cba6f",
            "ephemeral_id" : "0285f0b0-01e2-4691-9171-18e5a781cae2",
            "type" : "filebeat",
            "version" : "7.9.2"
          },
          "log" : {
            "file" : {
              "path" : "/var/log/syslog/name.log"
            },
            "original" : "%ASA-6-302014: Teardown TCP connection 23913857 for name-1:ip-2/49903 to name-2:ip-3/23 duration 25:54:08 bytes 4095435 TCP FINs",
            "offset" : 2207,
            "level" : "informational"
          },
          "message" : "Teardown TCP connection 23913857 for name-1:ip-2/49903 to name-2:ip-3/23 duration 25:54:08 bytes 4095435 TCP FINs",
          "fileset" : {
            "name" : "asa"
          },
          "error" : {
            "message" : [
              "Provided Grok expressions do not match field value: [Teardown TCP connection 23913857 for name-1:ip-2/49903 to name-2:ip-3/23 duration 25:54:08 bytes 4095435 TCP FINs]"
            ]
          },
          "tags" : [
            "cisco-asa",
            "forwarded"
          ],
          "input" : {
            "type" : "log"
          },
          "@timestamp" : "2020-11-03T23:59:04.000+01:00",
          "ecs" : {
            "version" : "1.5.0"
          },
          "service" : {
            "type" : "cisco"
          },
          "host" : {
            "hostname" : "name",
            "os" : {
              "kernel" : "value",
              "codename" : "Core",
              "name" : "CentOS Linux",
              "family" : "redhat",
              "version" : "7 (Core)",
              "platform" : "centos"
            },
            "containerized" : false,
            "ip" : [
              "ip-1"
            ],
            "id" : "41c28f6d08964fcab092cb65e2cc5c18",
            "mac" : [
              "00:00:00:00:00:00"
            ],
            "architecture" : "x86_64"
          },
          "event" : {
            "severity" : 6,
            "timezone" : "+01:00",
            "module" : "cisco",
            "action" : "flow-expiration",
            "dataset" : "cisco.asa"
          },
          "cisco" : {
            "asa" : {
              "message_id" : "302014"
            }
          }
        },
        "sort" : [
          1604444344000
        ]
      }
    ]
  }
}

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/1/c102324220a6030a5164cee2a9dd3e78c1a4ecf2.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/9/b9b35c0651d85804d7843f6967ed375e27995635.png)

I have just noticed that this probably is not taking form field I thought It was.

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [November 4, 2020, 3:33pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/4 "2020-11-04T15:33:02Z")

</div>

Oh wait, so that's an issue. Please, can you also open a GH issue here [https://github.com/elastic/beats/issues](https://github.com/elastic/beats/issues) so that someone fixes it?

By the way, about the yaml indentation, Filebeat won't give an error if your yaml is incorrect (correct YAML syntax but incorrect Filebeat I mean). It will simply think that the setting isn't there.

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [November 4, 2020, 3:40pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/5 "2020-11-04T15:40:40Z")

</div>

> [@Adriann](#):
>
> `"failed to find message"`

One last thing because after looking at this again, there's something I don't understand: Where do you get that message?

Your Cisco `message` is, for example,  
`Teardown TCP connection 23913857 for name-1:ip-2/49903 to name-2:ip-3/23 duration 25:54:08 bytes 4095435 TCP FINs`.

And your `error.message` like:  
`Provided Grok expressions do not match field value: [Teardown TCP connection 23913857 for name-1:ip-2/49903 to name-2:ip-3/23 duration 25:54:08 bytes 4095435 TCP FINs]`

So, for example, you can drop messages that `contains` [Define processors | Filebeat Reference [8.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html#condition-contains) an `error.message` field.

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [November 4, 2020, 3:51pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/6 "2020-11-04T15:51:36Z")

</div>

> [@Mario\_Castro](#):
>
> One last thing because after looking at this again, there's something I don't understand: Where do you get that message?

![image](https://us1.discourse-cdn.com/elastic/original/3X/f/a/fa97c29cf5125f1cdd59455743c7eebf5c5bae9a.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/e/fe7fb8318970b3d91fa85a1ec5156069e7ca62f9.png)

I have change It to

and added field in logs

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/4/044eb1e6692908722537e4bd90fb5ca5e95a4855.png)

```
processors:
  - drop_event:
         when:
         and:
         - equals.event.module: "cisco"
         - has_fields: ['error.message']

```

I don't get this output at all right now.

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [November 4, 2020, 3:57pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/7 "2020-11-04T15:57:12Z")

</div>

Sorry I didn't explain myself properly. I mean where the message is produced. Is this message produced in Filebeat Cisco module?

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [November 4, 2020, 3:59pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/8 "2020-11-04T15:59:53Z")

</div>

I am sending directly from filebeat to elastic.

As Kiban stands this "Message field is derived from document fields"

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/1/d1ce09e606a60d1a21e367617f9c4280b8a4c312.png)

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [November 4, 2020, 4:04pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/9 "2020-11-04T16:04:41Z")

</div>

> [@Adriann](#):
>
> ```auto
> processors:
> - drop_event:
> when:
> and:
> - equals.event.module: "cisco"
> - has_fields: ['error.message']
> 
> ```

It's working the last document with this field I got from

```
"@timestamp" : "2020-11-04T16:52:58.000+01:00",

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 2, 2020, 6:04pm UTC](https://discuss.elastic.co/t/how-to-drop-events-only-from-specific-module/254175/10 "2020-12-02T18:04:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.