Unable to exclude_lines in filebeat

Elastic Stack Beats
1

@warkolm
Hi.
My log line is
2019-01-22 10:25:01,401 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600
2019-01-22 10:25:01,401 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null.
2019-01-22 10:34:11,426 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()

I tried to below conf in filebeat.yml

exclude_lines: ["org.jboss.stdio.AbstractLoggingWriter.write"]
exclude_lines: ["AbstractLoggingWriter.write"]
exclude_lines: [".AbstractLoggingWriter.write."].

But I'm getting same errors in notification. Please help me how to drop above type log only. Please some one help me.
In filebeat I nead include error and warn but exclude above AbstractLoggingWriter.write.

Thanks,
Bala Melangi

2

@balamelangi,

Please try the below config in yout filebeat.yml

exclude_lines: ['^.*AbstractLoggingWriter.write..*$']

Thanks.

3

I'm having some issues getting filebeat to exclude lines. Here are the ways I've tried to use the regexp format with the exclude_lines: option in the apache2.yml file: ELK: start logstash fails prepaidgiftbalance illegalstateexception .

4

Thanks for your replay @Tek_Chand.
But still I'm getting notifications. My new config in filebeat is below.
exclude_lines: ['^.AbstractLoggingWriter.write..$']

I'm getting
2019-01-22 12:18:29,146 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) -
2019-01-22 12:18:30,312 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600
2019-01-22 12:15:25,786 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke()

If you have any other suggestions please help me.

Thanks.

5

my filebeat.yml file is :
filebeat.prospectors:

  • input_type: log
    paths:
    • /u04/jboss/standalone/log/server.log
    • /var/log/soapradius.log
    • /var/log/radius.log
      exclude_lines: ['^.AbstractLoggingWriter.write..$']

output.logstash:
hosts: ["x.x.x.x:5044"]
index: test

Please help me.

Thanks.

6

@balamelangi,

Please try below config:

exclude_lines: ['^.*AbstractLoggingWriter.*$']
7

Hi @Tek_Chand. But it's not working.
My new config in filebeat.yml is :
exclude_lines: ['^.AbstractLoggingWriter.$']

I'm getting below mail notification :

2019-01-22 12:48:54,060 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600
2019-01-22 12:48:54,060 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null 2019-01-22 12:48:54,514 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) -
2019-01-22 12:48:54,514 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null
2019-01-22 12:48:54,060 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke()

Thanks.

8

@here Is there any other way to drop above log lines using filebeat?

Thanks.

9

@balamelangi,

Can you please let me know from where you are getting this mail notification? Means which server is generating this mail?

Thanks.

10

Hi @Tek_Chand
I'm using beat as filebeat.
My architecture is below :

Filebeat---->Logstash---->ElasticSearch<-------Kibana.

In kibana I'm set notification when message match : "error" , Using X-pack notification.
In filebeat.yml I'm set logfile /u04/jboss/standalone/log/server.log .
I'm getting below log to mail when error match.

2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ServiceClass: org.apache.soap.providers.StatelessEJBProvider
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - Call.MethodName: nems
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()
2019-01-23 08:48:02,827 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600
2019-01-23 08:48:02,828 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke() .

My filebeat.yml file is
exclude_lines: ['^.AbstractLoggingWriter.']

11

@balamelangi,
are you sure that your filebeat.yml have below pattern for exclude line?

exclude_lines: ['^.AbstractLoggingWriter.]

But the pattern i have provided you is little bit different then above, which is look like below:

exclude_lines: ['^.*AbstractLoggingWriter.*$']
12

@Tek_Chand
I tried both. But not working.

Thanks.

13

@balamelangi,

you have set the above configuration on all filebeat server or only single server? May be you are receiving these logs from any other server.

I have used the same pattern at my end and its working fine.

14

@Tek_Chand
my watcher is

{
"trigger": {
"schedule": {
"interval": "2m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"platform.wavespot.net-*"
],
"types": ,
"body": {
"query": {
"bool": {
"must": {
"match": {
"message": "error"
}
},
"filter": {
"range": {
"@timestamp": {
"from": "now-2m",
"to": "now"
}
}
}
}
}
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 0
}
}
},
"actions": {
"email_admin": {
"email": {
"profile": "standard",
"to": [
"Bala Melangi bala.melangi@xxxxxx.com"
],
"subject": "Platform: JBOSS Errors from Watcher",
"body": {
"text": " Total {{ctx.payload.hits.total}} errors. Below are the errors from {{ctx.payload.hits.hits.0._source.beat.hostname}} \n {{ctx.payload.hits.hits.0._source.message}} \n {{ctx.payload.hits.hits.1._source.message}} \n {{ctx.payload.hits.hits.2._source.message}} \n {{ctx.payload.hits.hits.3._source.message}} \n {{ctx.payload.hits.hits.4._source.message}} \n {{ctx.payload.hits.hits.5._source.message}} \n {{ctx.payload.hits.hits.6._source.message}} \n {{ctx.payload.hits.hits.7._source.message}} \n {{ctx.payload.hits.hits.8._source.message}} \n"
}
}
}
}
}

15

@Tek_Chand I added the exclude_line to all my productions. Platform is one of the prod-server, And the server index is platform.wavespot.net-* please check.
I got error notification is,

Total 11 errors. Below are the errors from platform.wavespot.net
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()
2019-01-23 10:23:37,628 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - =============================================
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - Call.MethodName: nems
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - =============================================
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - URI: urn:pronto.ver600
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ServiceClass: org.apache.soap.providers.StatelessEJBProvider
2019-01-23 10:23:37,628 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.invoke()
2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - DD.ProviderClass: null
2019-01-23 10:22:48,233 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - Wed Jan 23 10:22:48 UTC 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

Thanks.

16

@balamelangi,

Can you please share you filebeat.yml file again. Please share it in formatted way so we can read it easily.

You can use </> to format your file.

Thanks.

17

@Tek_Chand
Please see my filebeat.yml file

filebeat.prospectors:

- input_type: log
  paths:
    - /u04/jboss/standalone/log/server.log
    - /var/log/soapradius.log
    - /var/log/radius.log
    exclude_lines: ['^.*AbstractLoggingWriter.*$']


output.logstash:
  hosts: ["x.x.x.x:5044"]
  index: platform.wavespot.net

Is there any other string I need to replace in place of AbstractLoggingWriter ?

Thanks.

18

Please format logs and config files using the </> button. Config files are sensitive to indentation (YAML file format). In your last example config the indentation of exclude_lines is off.

Tip: using ^.* or .*$ expresses a sub-string match. This is exactly what exclude_lines does. This config should work:

filebeat.prospectors:

- type: log
  paths:
    - /u04/jboss/standalone/log/server.log
    - /var/log/soapradius.log
    - /var/log/radius.log
  exclude_lines: ['AbstractLoggingWriter']


output.logstash:
  hosts: ["x.x.x.x:5044"]
  index: "platform.wavespot.net"
19

@steffens,

Same not working.

2019-01-23 10:23:37,627 ERROR stderr org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71) - In TemplateProvider.locate()

is my log.
Even I tried below
exclude_lines: ['^.AbstractLoggingWriter.$']
exclude_lines: ['AbstractLoggingWriter']
exclude_lines: ['^ERROR']
exclude_lines: ['ERROR']
exclude_lines: ['org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)']

exclude_lines: ['^.org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71).$']

Thanks.

20

@balamelangi,

Did you fix the identation issue in your filebeat.yml issue which is suggested in above post by Steffens?