# Can't drop events from metricbeat

**URL:** <https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [July 26, 2017, 9:12am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602 "2017-07-26T09:12:48Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [July 26, 2017, 9:12am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/1 "2017-07-26T09:12:48Z")

</div>

Hi folks,

We encountered an issue while the metricbeat is trying to send their metrics to elasticsearch.  
even with some drop events rules, but can't drop the events we don't want. we still can see some particular values of a field.

**metricbeat.yml**

```
 - drop_event:
      when:
         or:
           - equals: 
                system.filesystem.device_name: "tmpfs"
           - equals: 
                system.filesystem.device_name: "devpts"
           - equals: 
                system.filesystem.device_name: "none"
           - equals: 
                system.filesystem.device_name: "sysfs"
           - equals: 
                system.filesystem.device_name: "proc"
           - equals: 
                system.filesystem.device_name: "cgroup"
           - equals: 
                system.filesystem.device_name: "hugetlbfs"
           - equals: 
                system.filesystem.device_name: "systemd-1"
           - equals: 
                system.filesystem.device_name: "devtmpfs"
           - equals: 
                system.filesystem.device_name: "debugfs"
           - equals: 
                system.filesystem.mount_point: "dev/shm"
           - equals: 
                system.filesystem.mount_point: "/dev/pts"
           - equals: 
                system.filesystem.mount_point: "/sys/fs/selinux"
           - equals: 
                system.filesystem.mount_point: "/proc"
           - equals: 
                system.filesystem.mount_point: "/proc/*"
           - equals:
                sytem.filesystem.mount_point: "/dev"
           - equals:  
                sytem.filesystem.mount_point: "/dev/hugepages"
           - equals:  
                sytem.filesystem.mount_point: "/dev/mqueue"
           - equals:  
                sytem.filesystem.mount_point: "/proc/sys/fs/binfmt_misc"
           - equals: 
                sytem.filesystem.mount_point: "/sys/fs/cgroup/blkio"
           - equals:  
                sytem.filesystem.mount_point: "/sys/fs/cgroup/cpu,cpuacct"
           - equals:  
                sytem.filesystem.mount_point: "/sys/fs/cgroup/cpuset"
           - equals:  
               sytem.filesystem.mount_point: "/sys/fs/cgroup/devices"

```

**The results from grafana,but also can get the same results from kibana**

 ![](https://us1.discourse-cdn.com/elastic/original/3X/b/c/bc7a26046a7f9924cf7020330394b47104025416.png)

it would be great if there is a way can drop some events start with /sys/\*/.... with a wildcard.

Thanks

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [July 26, 2017, 9:27am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/2 "2017-07-26T09:27:05Z")

</div>

It's possible, I think you want to check [https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-processors.html#condition-regexp](https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-processors.html#condition-regexp)

We actually use it in system.yml example, check:

> <https://github.com/elastic/beats/blob/master/metricbeat/modules.d/system.yml#L24>

---

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [July 26, 2017, 10:03am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/3 "2017-07-26T10:03:55Z")

</div>

> [@exekias](#):
>
> sys|cgroup|proc|dev|etc|h

Hi @exekias,

with your instructions, we got more mount\_point that we expected.

**the metricbeat configure file**

**/etc/metricbeat/metricbeat.yml**

```
#================================ Processors ===================================
processors:
  - drop_fields:
      fields: ['metricset.rtt','beat.version','beat.hostname','_score','system.cpu.irq.pct','system.cpu.steal.pct','system.load.norm.1',
              'system.load.norm.5','system.load.norm.15','system.cpu.softirq.pct','system.memory.free','system.memory.used.pct',
              'system.memory.used.bytes','system.process.ppid','system.process.username','system.process.pgid','system.process.cpu.start_time']
      
  - drop_event.when.regexp:
      system.filesytem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

```

**the results from metricbeat debug mode**

```
 "dc": "fr4",
  "env": "production",
  "metricset": {
    "module": "system",
    "name": "filesystem"
  },
  "os": "linux",
  "service": "system",
  "system": {
    "filesystem": {
      "available": 0,
      "device_name": "cgroup",
      "files": 0,
      "free": 0,
      "free_files": 0,
      "mount_point": "/sys/fs/cgroup/memory",
      "total": 0,
      "used": {
        "bytes": 0,
        "pct": 0.000000
      }
    }
  },
  "tags": [
    "system"
  ],
  "type": "metricsets"
}
2017-07-26T17:57:50+08:00 DBG Publish: {
  "@timestamp": "2017-07-26T09:57:50.289Z",
  "agent": "metricbeat",
  "beat": {
    "name": "fr4eslogstash02"
  },
  "dc": "fr4",
  "env": "production",
  "metricset": {
    "module": "system",
    "name": "filesystem"
  },
  "os": "linux",
  "service": "system",
  "system": {
    "filesystem": {
      "available": 0,
      "device_name": "cgroup",
      "files": 0,
      "free": 0,
      "free_files": 0,
      "mount_point": "/sys/fs/cgroup/cpuset",

```

**the results from grafana dashboard.**

 ![](https://us1.discourse-cdn.com/elastic/original/3X/b/3/b3f47cb84810a23975ffae69c505774536c1e915.png)

---

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [July 26, 2017, 10:26am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/4 "2017-07-26T10:26:09Z")

</div>

**Any ideas on this issue?**

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [July 26, 2017, 10:32am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/5 "2017-07-26T10:32:32Z")

</div>

uhm, settings look good to me, did you restart metrcibeat after updating them?

---

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [July 26, 2017, 10:41am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/6 "2017-07-26T10:41:25Z")

</div>

I did this, and also restart the logstash..  
I'm using the metricbeat 5.4.3...

a complate the drop events as following:

```
- drop_event:
      when:
        or:
          - regexp:
              system.filesytem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib|run)'
          - regexp:
              system.filesytem.device_name: '^(selinuxfs|systemd-1|mqueue|debugfs|hugetlbfs|binfmt_misc|tmpfs|sysfs|proc|devtmpfs|securityfs|devpts|cgroup|pstore|configfs)'
```

---

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [July 26, 2017, 11:10am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/7 "2017-07-26T11:10:34Z")

</div>

> [@exekias](#):
>
> pdating them?

I used centos 7 with metricbeat version v5.4.3.

whatever any methods I tried, it doesn't works with metricbeat drop event rule.

😭 😭 😭 😭

---

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [July 26, 2017, 11:11am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/8 "2017-07-26T11:11:33Z")

</div>

@exekias  
Can you give me more options or more drop events rule?

---

<div class="post-metadata">

**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\
**Post date:** [July 27, 2017, 9:40am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/9 "2017-07-27T09:40:30Z")

</div>

Just to troubleshoot, can you start with a regexp that will match almost anything, just to make sure the processors are applied? Something like:

```auto
processors:
- drop_event.when.regexp:
    system.filesystem.mount_point: '.*'

```

Does this drop the events for you?

---

<div class="post-metadata">

**Author:** ![Robin\_Guo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robin_guo/32/42297_2.png) [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\
**Post date:** [July 31, 2017, 4:15am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/10 "2017-07-31T04:15:03Z")

</div>

> [@tudor](#):
>
> system.filesystem.mount\_

hi @tudor,

Finally, the problem is fixed with filters or processors to drop the events.

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 28, 2017, 4:15am UTC](https://discuss.elastic.co/t/cant-drop-events-from-metricbeat/94602/11 "2017-08-28T04:15:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.