# Block certain strings from ingestion

**URL:** <https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326>\
**Category:** Elasticsearch\
**Created:** [January 2, 2023, 3:46pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326 "2023-01-02T15:46:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rahul\_sirugudi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahul_sirugudi/32/94019_2.png) [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)\
**Post date:** [January 2, 2023, 3:46pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/1 "2023-01-02T15:46:41Z")

</div>

Hi,

There are few back end applications which depend/listen to queues, if there are no messages in queue it will print as INFO level logs "NO MESSAGES AVAILABLE in queue". Is there a way where i can block this particular string to get ingested to Elasticsearch?

this is my flow looks like.  
beats =\> logstash =\> es

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [January 2, 2023, 6:11pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/2 "2023-01-02T18:11:02Z")

</div>

Hi @rahul_sirugudi

Perhaps look at filebeat [drop event processor](https://www.elastic.co/guide/en/beats/filebeat/current/drop-event.html)

Or perhaps look at one of the many logstash topics on this...

> [@Drop the complete message containing specific strings](https://discuss.elastic.co/t/drop-the-complete-message-containing-specific-strings/197799/1):
>
> Hello, I have the following definition to drop messages from a log file containing strings and text: input { file { path =\> "/opt/mapr/logs/cldb.log" tags =\> "mapr\_cldb" codec =\> plain {charset =\> "ISO-8859-1"} } } filter { if "INFO" in [message] { drop{ } } if "[CLDB-1]:" in [message] { drop{ } } if "reqIncoming" in [message] { drop{ } } if "The server has decided to close" in [message] { drop{ } } if "WARN log" in [message] { drop{ } } if "RpcProgram not found" in [message] { dr…

---

<div class="post-metadata">

**Author:** ![rahul\_sirugudi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahul_sirugudi/32/94019_2.png) [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)\
**Post date:** [January 6, 2023, 11:59am UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/3 "2023-01-06T11:59:51Z")

</div>

Thanks, i tried from beats but now all the logs are blocked.

```auto
input {
  beats {
    port => 5044
    ssl => false
  }
}
match => { "message" => "%{IPV4:ip} - \[%{TIMESTAMP_ISO8601:timestamp}\] - %{GREEDYDATA:message} - %{GREEDYDATA:pool} - %{LOGLEVEL:log-level} : %{GREEDYDATA:error-message}" }
output {
if [fields][type] == "test"
{
if "NO MESSAGES AVAILABLE in queue..." in [error-message] { drop { } }
{
stdout { codec => rubydebug }
elasticsearch {
hosts => ["0.0.0.0:9200"]
   user => "username"
   password => "password"
   index => "index"
}
}
}
}

```

```auto

```

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 6, 2023, 12:37pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/4 "2023-01-06T12:37:05Z")

</div>

@rahul_sirugudi Can you share what have you tried on beats to drop the events ?

---

<div class="post-metadata">

**Author:** ![rahul\_sirugudi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahul_sirugudi/32/94019_2.png) [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)\
**Post date:** [January 6, 2023, 12:41pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/5 "2023-01-06T12:41:17Z")

</div>

oops my bad, i meant in the logstash beats config i made changes. `{ drop { } }` . My guess is this will drop the mathced string.

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [January 6, 2023, 1:32pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/6 "2023-01-06T13:32:38Z")

</div>

@rahul_sirugudi I believe what you want to do can be done before even sending the event to logstash. As suggested by @stephenb please use the drop\_event processor of filebeat putting in the required conditions - either equals or contains can be used in your case.

---

<div class="post-metadata">

**Author:** ![rahul\_sirugudi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahul_sirugudi/32/94019_2.png) [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)\
**Post date:** [January 6, 2023, 1:36pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/7 "2023-01-06T13:36:01Z")

</div>

i am able to achieve this now thanks. My bad i should have kept my condition just after grok under filter section.

```auto
filter {
grok{
match => { "message" => "%{IPV4:ip} - \[%{TIMESTAMP_ISO8601:timestamp}\] - %{GREEDYDATA:message} - %{GREEDYDATA:pool} - %{LOGLEVEL:log-level} : %{GREEDYDATA:error-message}" }
}
if "NO MESSAGES AVAILABLE in queue..." in [error-message] { drop { } }
}

```

this helped me avoiding the ingesting messages.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 3, 2023, 1:36pm UTC](https://discuss.elastic.co/t/block-certain-strings-from-ingestion/322326/8 "2023-02-03T13:36:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.