# How we can drop log line using filebeat **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [November 2, 2018, 10:17am UTC](https://discuss.elastic.co/t/how-we-can-drop-log-line-using-filebeat/155147 "2018-11-02T10:17:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Post date:** [November 2, 2018, 10:17am UTC](https://discuss.elastic.co/t/how-we-can-drop-log-line-using-filebeat/155147/1 "2018-11-02T10:17:14Z")
Hello Team, I am aware how we can exclude a particular log type (line) using filebeat and i have implemented it and its working fine. But now i am getting 20 line of same log type and i want to exclude 19 of them at filebeat level and want to send only one line to elasticserach. Below are the the sample log line: ```auto Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz Nov 2 09:46:44 xyz sshd[32511]: Accepted publickey for xyz ``` is it possible to drop all logs line except one? I have tried but that drops all lines. Any assistance will be appreciated. Thanks. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [November 2, 2018, 12:15pm UTC](https://discuss.elastic.co/t/how-we-can-drop-log-line-using-filebeat/155147/2 "2018-11-02T12:15:06Z")
Hi @Tek_Chand, Currently there is no way to remove duplications from filebeat, and in general each event is independent to the previous ones. Removing duplications can be too dependant of the use case. I guess it could be possible to implement something like multiline, but to detect consecutive duplicated log lines. A field could be added with the number of repetitions too. You can [open an issue](https://github.com/elastic/beats/issues/new) with this feature request. ---
**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Post date:** [November 12, 2018, 4:13am UTC](https://discuss.elastic.co/t/how-we-can-drop-log-line-using-filebeat/155147/3 "2018-11-12T04:13:36Z")
@Jaime, Thank you for your response. I have open a issue at Github. Below is the id of issue: [[http://Duplication](http://Duplication) removal at filebeat level. #9033]([http://Duplication](http://Duplication) removal at filebeat level. #9033) Thanks. ---
**Author:** ![bigphil](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@bigphil](https://discuss.elastic.co/u/bigphil)\ **Post date:** [November 12, 2018, 4:32am UTC](https://discuss.elastic.co/t/how-we-can-drop-log-line-using-filebeat/155147/4 "2018-11-12T04:32:45Z")
I know it's not quite the answer you're looking for, but if you can first send the events to a Logstash node, you could use the throttle filter or even the drop filter with percentage option. ---
**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Post date:** [November 12, 2018, 5:01am UTC](https://discuss.elastic.co/t/how-we-can-drop-log-line-using-filebeat/155147/5 "2018-11-12T05:01:26Z")
@Philip, Thank you for your suggestion. I will read about throttle and drop filter with percentage. Thanks. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 10, 2018, 5:01am UTC](https://discuss.elastic.co/t/how-we-can-drop-log-line-using-filebeat/155147/6 "2018-12-10T05:01:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.