# Filter or remove some logs in logstash

**URL:** <https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519>\
**Category:** Logstash\
**Created:** [October 23, 2018, 4:37am UTC](https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519 "2018-10-23T04:37:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![prem\_kumar](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@prem\_kumar](https://discuss.elastic.co/u/prem_kumar)\
**Post date:** [October 23, 2018, 4:37am UTC](https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519/1 "2018-10-23T04:37:46Z")

</div>

I have Filebeat -\> logstash -\> Elasticsearch setup and everything is working fine.  
I have filter condition in logstash so that I can parse.  
In my log file it will have some additional logs lines like

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

- Server started +  
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

I can see these logs in elasticserach. I don't want these logs to be in elasticsearch.  
Is there anyway so the I can only send particular logs to logstash.

---

<div class="post-metadata">

**Author:** ![bloke](https://avatars.discourse-cdn.com/v4/letter/b/b5e925/32.png) [@bloke](https://discuss.elastic.co/u/bloke)\
**Post date:** [October 23, 2018, 5:33am UTC](https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519/2 "2018-10-23T05:33:03Z")

</div>

Look further into your filebeat.yml setup

in particular the paths: section you need to direct the Prospector in finer detail

for example if you want to exclude a log file type you can use wildcards

```
- type: log

  # Change to true to enable this prospector configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    #- /var/log/*.log
    - d:\logs\iamalog*.log
```

---

<div class="post-metadata">

**Author:** ![prem\_kumar](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@prem\_kumar](https://discuss.elastic.co/u/prem_kumar)\
**Post date:** [October 23, 2018, 5:46am UTC](https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519/3 "2018-10-23T05:46:33Z")

</div>

Thanks from the response @bloke but my issue is different.  
I have a log file, inside that log file I have some log lines as I have mentioned above how to remove that.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 23, 2018, 5:58am UTC](https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519/4 "2018-10-23T05:58:54Z")

</div>

You might be able to use a [drop\_event processor](https://www.elastic.co/guide/en/beats/filebeat/6.4/drop-event.html) in Filebeat or a [drop filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-drop.html) in Logstash.

---

<div class="post-metadata">

**Author:** ![bloke](https://avatars.discourse-cdn.com/v4/letter/b/b5e925/32.png) [@bloke](https://discuss.elastic.co/u/bloke)\
**Post date:** [October 23, 2018, 5:59am UTC](https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519/5 "2018-10-23T05:59:34Z")

</div>

use the exclude feature in the prospector, within filebeat.yml

```
  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  exclude_lines: ['^DBG']

```

and match your lines with a regex tool like [https://regex101.com/](https://regex101.com/)

hope it helps

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 20, 2018, 5:59am UTC](https://discuss.elastic.co/t/filter-or-remove-some-logs-in-logstash/153519/6 "2018-11-20T05:59:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.