# Filestream exclude\_lines doesn't appear to be working

**URL:** <https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759>\
**Category:** Elastic Agent\
**Tags:** filebeat\
**Created:** [June 19, 2024, 8:53pm UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759 "2024-06-19T20:53:41Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Steve\_Foster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_foster/32/125583_2.png) [@Steve\_Foster](https://discuss.elastic.co/u/Steve_Foster)\
**Post date:** [June 19, 2024, 8:53pm UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/1 "2024-06-19T20:53:41Z")

</div>

I'm trying to stop the inclusion of healthcheck log lines from being sent to Elasticsearch.

In the filestream [docs](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#filebeat-input-filestream-exclude-lines) there is the exclude\_lines option which is an array of regex.

So if I want to filter out lines that contain /healthz or /ping then I should add in

```auto
exclude_lines: ['/healthz', '/ping']

```

into the "Custom configuration" in the "Collect kubernetes container logs" section in the Kubernetes integration config for the agent policy. Like the following?

 ![Edit-integration-MWS-test-Agent-policies-Fleet-Elastic](https://us1.discourse-cdn.com/elastic/original/3X/3/b/3bfb9b7f07d179b404f7b02a39c403a1553df1db.png)

But I still see them coming through from the agent.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 23, 2024, 3:55am UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/2 "2024-06-23T03:55:40Z")

</div>

Hi @Steve_Foster

Not a RegExpert.. but Did you try...

`exclude_lines: ['\/healthz', '\/ping']`

---

<div class="post-metadata">

**Author:** ![Steve\_Foster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_foster/32/125583_2.png) [@Steve\_Foster](https://discuss.elastic.co/u/Steve_Foster)\
**Post date:** [June 23, 2024, 10:40am UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/3 "2024-06-23T10:40:44Z")

</div>

Hi @stephenb

yeah that was my second thought...

![image](https://us1.discourse-cdn.com/elastic/original/3X/9/8/9865c47b7f3d410976789072bb147a0719cd2533.png)

but that doesn't work either.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 23, 2024, 1:13pm UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/4 "2024-06-23T13:13:39Z")

</div>

Can you please share a couple lines of good vs to exclude

---

<div class="post-metadata">

**Author:** ![Steve\_Foster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_foster/32/125583_2.png) [@Steve\_Foster](https://discuss.elastic.co/u/Steve_Foster)\
**Post date:** [June 23, 2024, 8:01pm UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/5 "2024-06-23T20:01:19Z")

</div>

```auto
{"content_length":0,"http_version":"1.1","level":"info","method":"GET","response_time":0.0162,"status":204,"timestamp":"2024-06-23T19:57:49.261Z","url":"/healthz","user_agent":"kube-probe/1.28+"}
{"content_length":0,"http_version":"1.1","level":"info","method":"GET","response_time":0.02162,"status":204,"timestamp":"2024-06-23T19:57:59.259Z","url":"/healthz","user_agent":"kube-probe/1.28+"}
{"content_length":0,"http_version":"1.1","level":"info","method":"GET","response_time":0.013789,"status":204,"timestamp":"2024-06-23T19:57:59.260Z","url":"/healthz","user_agent":"kube-probe/1.28+"}
{"content_length":0,"http_version":"1.1","level":"info","method":"GET","response_time":0.021483,"status":204,"timestamp":"2024-06-23T19:58:09.260Z","url":"/healthz","user_agent":"kube-probe/1.28+"}
{"content_length":0,"http_version":"1.1","level":"info","method":"GET","response_time":0.017812,"status":204,"timestamp":"2024-06-23T19:58:09.261Z","url":"/healthz","user_agent":"kube-probe/1.28+"}

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 23, 2024, 10:16pm UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/6 "2024-06-23T22:16:03Z")

</div>

I think the perhaps the issue is it's going through the [json parser](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#filebeat-input-filestream-ndjson) first so it's extracting the ndjson first before exclude... But I can't say for sure....Not at my desk at the moment...

> The decoding happens before line filtering. You can combine JSON decoding with filtering if you set the message\_key option. This can be helpful in situations where the application logs are wrapped in JSON objects, like when using Docker.

So perhaps use drop event processor and see if that works (it will).

Probably easy to test. Take out the json parser and see if it drops the lines,..

---

<div class="post-metadata">

**Author:** ![Steve\_Foster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_foster/32/125583_2.png) [@Steve\_Foster](https://discuss.elastic.co/u/Steve_Foster)\
**Post date:** [June 24, 2024, 2:22am UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/7 "2024-06-24T02:22:11Z")

</div>

Yeah `drop_event` works as expected... Thanks

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 24, 2024, 2:28pm UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/8 "2024-06-24T14:28:11Z")

</div>

@Steve_Foster, Can you share your `drop_event` code in case anyone else ends up in this topic? Thanks.

---

<div class="post-metadata">

**Author:** ![Steve\_Foster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_foster/32/125583_2.png) [@Steve\_Foster](https://discuss.elastic.co/u/Steve_Foster)\
**Post date:** [June 24, 2024, 9:41pm UTC](https://discuss.elastic.co/t/filestream-exclude-lines-doesnt-appear-to-be-working/361759/9 "2024-06-24T21:41:29Z")

</div>

Sure thing

```yaml
- drop_event:
    when:
      or:
        - equals:
            RequestPath: "/ping"
        - equals:
            url: "/healthz"

```

I suspect that I will find some others over time and add them to this list