# Unable to exclude\_lines in filebeat

**URL:** <https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 22, 2019, 10:36am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215 "2019-01-22T10:36:40Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 22, 2019, 10:36am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/1 "2019-01-22T10:36:41Z")

</div>

@warkolm  
Hi.  
My log line is  
2019-01-22 10:25:01,401 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600  
2019-01-22 10:25:01,401 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null.  
2019-01-22 10:34:11,426 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()

I tried to below conf in filebeat.yml

exclude\_lines: ["org.jboss.stdio.AbstractLoggingWriter.write"]  
exclude\_lines: ["AbstractLoggingWriter.write"]  
exclude\_lines: ["._AbstractLoggingWriter.write._"].

But I'm getting same errors in notification. Please help me how to drop above type log only. Please some one help me.  
In filebeat I nead **include**  **error** and **warn** but **exclude** above **AbstractLoggingWriter.write**.

Thanks,  
Bala Melangi

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [January 22, 2019, 10:52am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/2 "2019-01-22T10:52:06Z")

</div>

@balamelangi,

Please try the below config in yout `filebeat.yml`

```auto
exclude_lines: ['^.*AbstractLoggingWriter.write..*$']

```

Thanks.

---

<div class="post-metadata">

**Author:** ![Jorja073](https://avatars.discourse-cdn.com/v4/letter/j/9fc29f/32.png) [@Jorja073](https://discuss.elastic.co/u/Jorja073)\
**Post date:** [January 22, 2019, 11:44am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/3 "2019-01-22T11:44:11Z")

</div>

I'm having some issues getting filebeat to exclude lines. Here are the ways I've tried to use the regexp format with the exclude\_lines: option in the apache2.yml file: ELK: start logstash fails [prepaidgiftbalance](https://prepaidgiftbalance.net/) illegalstateexception .

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 22, 2019, 12:26pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/4 "2019-01-22T12:26:28Z")

</div>

Thanks for your replay @Tek_Chand.  
But still I'm getting notifications. My new config in filebeat is below.  
exclude\_lines: ['^._AbstractLoggingWriter.write.._$']

I'm getting  
2019-01-22 12:18:29,146 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) -  
2019-01-22 12:18:30,312 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600  
2019-01-22 12:15:25,786 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke()

If you have any other suggestions please help me.

Thanks.

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 22, 2019, 12:31pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/5 "2019-01-22T12:31:00Z")

</div>

my filebeat.yml file is :  
filebeat.prospectors:

- input\_type: log  
paths:
  - /u04/jboss/standalone/log/server.log
  - /var/log/soapradius.log
  - /var/log/radius.log  
exclude\_lines: ['^._AbstractLoggingWriter.write.._$']

output.logstash:  
hosts: ["x.x.x.x:5044"]  
index: test

Please help me.

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [January 22, 2019, 12:31pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/6 "2019-01-22T12:31:36Z")

</div>

@balamelangi,

Please try below config:

```auto
exclude_lines: ['^.*AbstractLoggingWriter.*$']

```

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 22, 2019, 12:51pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/7 "2019-01-22T12:51:59Z")

</div>

Hi @Tek_Chand. But it's not working.  
My new config in filebeat.yml is :  
exclude\_lines: ['^._AbstractLoggingWriter._$']

I'm getting below mail notification :

2019-01-22 12:48:54,060 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600  
2019-01-22 12:48:54,060 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null 2019-01-22 12:48:54,514 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) -  
2019-01-22 12:48:54,514 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null  
2019-01-22 12:48:54,060 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke()

Thanks.

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 22, 2019, 12:54pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/8 "2019-01-22T12:54:00Z")

</div>

@here Is there any other way to drop above log lines using filebeat?

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [January 23, 2019, 9:37am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/9 "2019-01-23T09:37:03Z")

</div>

@balamelangi,

> [@balamelangi](#):
>
> I'm getting below mail notification

Can you please let me know from where you are getting this mail notification? Means which server is generating this mail?

Thanks.

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 23, 2019, 9:48am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/10 "2019-01-23T09:48:58Z")

</div>

Hi @Tek_Chand  
I'm using beat as filebeat.  
My architecture is below :

Filebeat----\>Logstash----\>ElasticSearch\<-------Kibana.

In kibana I'm set notification when message match : "error" , Using X-pack notification.  
In filebeat.yml I'm set logfile **/u04/jboss/standalone/log/server.log**.  
I'm getting below log to mail when **error** match.

2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null  
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ServiceClass: org.apache.soap.providers.StatelessEJBProvider  
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - Call.MethodName: nems  
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()  
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600  
2019-01-23 08:48:02,828 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke() .

My filebeat.yml file is  
exclude\_lines: ['^._AbstractLoggingWriter._']

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [January 23, 2019, 10:06am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/11 "2019-01-23T10:06:56Z")

</div>

@balamelangi,  
are you sure that your `filebeat.yml` have below pattern for exclude line?

```auto
exclude_lines: ['^.AbstractLoggingWriter.]

```

But the pattern i have provided you is little bit different then above, which is look like below:

```auto
exclude_lines: ['^.*AbstractLoggingWriter.*$']

```

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 23, 2019, 10:08am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/12 "2019-01-23T10:08:52Z")

</div>

@Tek_Chand  
I tried both. But not working.

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [January 23, 2019, 10:10am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/13 "2019-01-23T10:10:43Z")

</div>

@balamelangi,

you have set the above configuration on all filebeat server or only single server? May be you are receiving these logs from any other server.

I have used the same pattern at my end and its working fine.

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 23, 2019, 10:20am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/14 "2019-01-23T10:20:39Z")

</div>

@Tek_Chand  
my watcher is

{  
"trigger": {  
"schedule": {  
"interval": "2m"  
}  
},  
"input": {  
"search": {  
"request": {  
"search\_type": "query\_then\_fetch",  
"indices": [  
"platform.wavespot.net-\*"  
],  
"types": ,  
"body": {  
"query": {  
"bool": {  
"must": {  
"match": {  
"message": "error"  
}  
},  
"filter": {  
"range": {  
"@timestamp": {  
"from": "now-2m",  
"to": "now"  
}  
}  
}  
}  
}  
}  
}  
}  
},  
"condition": {  
"compare": {  
"ctx.payload.hits.total": {  
"gt": 0  
}  
}  
},  
"actions": {  
"email\_admin": {  
"email": {  
"profile": "standard",  
"to": [  
"Bala Melangi [bala.melangi@xxxxxx.com](mailto:bala.melangi@xxxxxx.com)"  
],  
"subject": "Platform: JBOSS Errors from Watcher",  
"body": {  
"text": " Total {{ctx.payload.hits.total}} errors. Below are the errors from {{ctx.payload.hits.hits.0.\_source.beat.hostname}} \n {{ctx.payload.hits.hits.0.\_source.message}} \n {{ctx.payload.hits.hits.1.\_source.message}} \n {{ctx.payload.hits.hits.2.\_source.message}} \n {{ctx.payload.hits.hits.3.\_source.message}} \n {{ctx.payload.hits.hits.4.\_source.message}} \n {{ctx.payload.hits.hits.5.\_source.message}} \n {{ctx.payload.hits.hits.6.\_source.message}} \n {{ctx.payload.hits.hits.7.\_source.message}} \n {{ctx.payload.hits.hits.8.\_source.message}} \n"  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 23, 2019, 10:24am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/15 "2019-01-23T10:24:59Z")

</div>

@Tek_Chand I added the **exclude\_line** to all my productions. Platform is one of the prod-server, And the server index is **platform.wavespot.net-** \* please check.  
I got error notification is,

Total 11 errors. Below are the errors from [platform.wavespot.net](http://platform.wavespot.net)  
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()  
2019-01-23 10:23:37,628 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - =============================================  
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - Call.MethodName: nems  
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - =============================================  
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600  
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ServiceClass: org.apache.soap.providers.StatelessEJBProvider  
2019-01-23 10:23:37,628 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke()  
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null  
2019-01-23 10:22:48,233 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - Wed Jan 23 10:22:48 UTC 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [January 23, 2019, 10:31am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/16 "2019-01-23T10:31:57Z")

</div>

@balamelangi,

Can you please share you `filebeat.yml` file again. Please share it in formatted way so we can read it easily.

You can use \</\> to format your file.

Thanks.

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 23, 2019, 10:42am UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/17 "2019-01-23T10:42:20Z")

</div>

@Tek_Chand  
Please see my filebeat.yml file

```auto
filebeat.prospectors:

- input_type: log
  paths:
    - /u04/jboss/standalone/log/server.log
    - /var/log/soapradius.log
    - /var/log/radius.log
    exclude_lines: ['^.*AbstractLoggingWriter.*$']

output.logstash:
  hosts: ["x.x.x.x:5044"]
  index: platform.wavespot.net

```

Is there any other string I need to replace in place of **AbstractLoggingWriter**?

Thanks.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 23, 2019, 12:14pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/18 "2019-01-23T12:14:46Z")

</div>

Please format logs and config files using the `</>` button. Config files are sensitive to indentation (YAML file format). In your last example config the indentation of `exclude_lines` is off.

Tip: using `^.*` or `.*$` expresses a sub-string match. This is exactly what exclude\_lines does. This config should work:

```auto
filebeat.prospectors:

- type: log
  paths:
    - /u04/jboss/standalone/log/server.log
    - /var/log/soapradius.log
    - /var/log/radius.log
  exclude_lines: ['AbstractLoggingWriter']

output.logstash:
  hosts: ["x.x.x.x:5044"]
  index: "platform.wavespot.net"

```

---

<div class="post-metadata">

**Author:** ![balamelangi](https://avatars.discourse-cdn.com/v4/letter/b/c77e96/32.png) [@balamelangi](https://discuss.elastic.co/u/balamelangi)\
**Post date:** [January 23, 2019, 12:26pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/19 "2019-01-23T12:26:49Z")

</div>

@steffens,

Same not working.

2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()

is my log.  
Even I tried below  
exclude\_lines: ['^._AbstractLoggingWriter._$']  
exclude\_lines: ['AbstractLoggingWriter']  
exclude\_lines: ['^ERROR']  
exclude\_lines: ['ERROR']  
exclude\_lines: ['org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)']

exclude\_lines: ['^._org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)._$']

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [January 23, 2019, 12:31pm UTC](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215/20 "2019-01-23T12:31:13Z")

</div>

@balamelangi,

> [@balamelangi](#):
>
> Same not working.

Did you fix the identation issue in your `filebeat.yml` issue which is suggested in above post by Steffens?

> [@steffens](#):
>
> In your last example config the indentation of `exclude_lines` is off.

[Next page](https://discuss.elastic.co/t/unable-to-exclude-lines-in-filebeat/165215.md?page=2)