# Unable to remove lines from message filed **URL:** \ **Category:** Logstash\ **Created:** [April 7, 2022, 12:50pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866 "2022-04-07T12:50:22Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![amsmzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amsmzn/32/104115_2.png) [@amsmzn](https://discuss.elastic.co/u/amsmzn)\ **Post date:** [April 7, 2022, 12:50pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/1 "2022-04-07T12:50:23Z")
I am trying to remove the below lines from my jenkins build logs with mutate, gsub filter but unable to do so. Can anybody please help me in this. Downloaded from snapshots: [https://abc.com/abc.pom](https://abc.com/abc.pom) **16:39:44** Progress (1): 2.6 kB ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 8, 2022, 2:16pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/2 "2022-04-08T14:16:29Z")
If you want to delete events when they match a pattern then use drop. ``` if [message] =~ /SomePatternHere/ { drop {} } ``` ---
**Author:** ![amsmzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amsmzn/32/104115_2.png) [@amsmzn](https://discuss.elastic.co/u/amsmzn)\ **Post date:** [April 8, 2022, 2:49pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/3 "2022-04-08T14:49:14Z")
I tried below filter: if [message] = "Downloaded" { drop {} } But didnt work for me. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 8, 2022, 3:21pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/4 "2022-04-08T15:21:37Z")
> [@amsmzn](#): > > if [message] = "Downloaded" { drop {} } If you want to test whether the [message] field is exactly equal to "Downloaded" then use ``` if [message] == "Downloaded" ``` If you want to test whether the [message] field contains the word "Downloaded" you can use either ``` if "Downloaded" in [message] ``` or ``` if [message] =~ /Downloaded/ ``` ---
**Author:** ![amsmzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amsmzn/32/104115_2.png) [@amsmzn](https://discuss.elastic.co/u/amsmzn)\ **Post date:** [April 9, 2022, 11:24am UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/5 "2022-04-09T11:24:10Z")
Thanks Badger for your response. Below is my config file: ``` input { beats { port => 5044 } } filter { prune { blacklist_names => [" \e\[8mha.*\e\[0m "] } } filter { mutate { gsub => ["message", "\e\[8mha.*\e\[0m", ""] } } filter { if [message] =~ /Downloaded/ { drop { } } } ``` I have introduced drop filter but now i am not getting any logs on the kibana ui. Same is with this if "Downloaded" in [message] . ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 9, 2022, 5:36pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/6 "2022-04-09T17:36:44Z")
> [@amsmzn](#): > > I have introduced drop filter but now i am not getting any logs on the kibana ui. That would suggest that all of your messages contain "Downloaded". ---
**Author:** ![amsmzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amsmzn/32/104115_2.png) [@amsmzn](https://discuss.elastic.co/u/amsmzn)\ **Post date:** [April 9, 2022, 5:51pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/7 "2022-04-09T17:51:16Z")
I want to remove these lines from the logs. Since they are repetitive. Downloaded from snapshots: [https://abc.com/abc.pom](https://abc.com/abc.pom) **16:39:44** Progress (1): 2.6 kB I don't want the complete logs to get dropped. Please suggest. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 9, 2022, 6:13pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/8 "2022-04-09T18:13:35Z")
The conditional around the drop filter means only lines containing the word "Downloaded" will get dropped. Without seeing the complete data set that your beat is sending there is no way to know why that might result in no data reaching kibana. ---
**Author:** ![amsmzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amsmzn/32/104115_2.png) [@amsmzn](https://discuss.elastic.co/u/amsmzn)\ **Post date:** [April 10, 2022, 4:10pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/9 "2022-04-10T16:10:07Z")
It should have dropped just those lines with "Downloaded" word, but its dropping the complete logs. I tried to remove it with mutate filter. Below is the configuration: filter { mutate { gsub =\> ["message", "Downloaded from snapshots: [https://abc.com](https://abc.com).\*", ""] } } This works but it is leaving blank spaces in the logs. Like there are bunch of empty lines showing between two readable lines. Could you please suggest how can i remove empty lines from the logs because drop filter just doesn't work for me. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 10, 2022, 5:44pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/10 "2022-04-10T17:44:03Z")
> [@amsmzn](#): > > Could you please suggest how can i remove empty lines from the logs Not without seeing the actual data set you are processing. What you tried should have worked. ---
**Author:** ![amsmzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amsmzn/32/104115_2.png) [@amsmzn](https://discuss.elastic.co/u/amsmzn)\ **Post date:** [April 11, 2022, 6:04am UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/11 "2022-04-11T06:04:00Z")
Below are the lines which are repetitve in logs: Downloaded from snapshots: [https://hc-us-east-aws-artifactory.abc.com/artifactory/maven-xyz/org/xyz/plexus/abc-components/1.1.6/abc-components-1.1.6.pom](https://hc-us-east-aws-artifactory.abc.com/artifactory/maven-xyz/org/xyz/plexus/abc-components/1.1.6/abc-components-1.1.6.pom) (1.9 kB at 26 kB/s) 16:39:46 Downloading from snapshots: [https://hc-us-east-aws-artifactory.abc.com/artifactory/maven-xyz/org/xyz/plexus/abc/1.0.8/plexus-1.0.8.pom](https://hc-us-east-aws-artifactory.abc.com/artifactory/maven-xyz/org/xyz/plexus/abc/1.0.8/plexus-1.0.8.pom) 16:39:46 Progress (1): 3.4/6.6 kB Progress (1): 6.6 kB After removing them with mutate gsub, they are leaving behind empty lines ---
**Author:** ![amsmzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amsmzn/32/104115_2.png) [@amsmzn](https://discuss.elastic.co/u/amsmzn)\ **Post date:** [April 11, 2022, 12:31pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/12 "2022-04-11T12:31:27Z")
I have fixed this. filter { mutate { gsub =\> ["message", "\[1\]\*$", ""] } } One can remove empty lines with the above filter. Thanks for the assisstance. * * * 1. [:space:] ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 9, 2022, 12:31pm UTC](https://discuss.elastic.co/t/unable-to-remove-lines-from-message-filed/301866/13 "2022-05-09T12:31:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.