Issues with regexes and kubernetes hints based autodiscovery with filebeat 6.4.0


(Alex Scoble) #1

What I'm about to describe works fine in filebeat 6.3.1. The below only happens when I upgrade the filebeat daemonset for kubernetes to 6.4.0.

We are using hints based autodiscovery in kubernetes. The pod annotations are typically something like

  annotations:
    co.elastic.logs/exclude_lines: 'GET\s\"?/status|^\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)[m|K]\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)[m|K]$'
    co.elastic.logs.service-foo/multiline.pattern: '^(\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K](\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K])?)?([0-9]{2}:[0-9]{2}|<[0-9]{2}>[0-9]\s[0-9]{4}-[0-9]{2}-[0-9]{2})'
    co.elastic.logs.service-foo/multiline.negate: 'true'
    co.elastic.logs.service-foo/multiline.match: 'after'
    co.elastic.logs.service-foo-nginx/module: 'nginx'
    co.elastic.logs.service-foo-nginx/fileset.stdout: 'access'
    co.elastic.logs.service-foo-nginx/fileset.stderr: 'error'

The errors we are getting in filebeat 6.4.0 are like below:

2018-09-11T22:22:00.369Z	ERROR	fileset/factory.go:105	Error creating input: error parsing regexp: missing closing ): `GET\s"?/status|^\x1B\[([0-9]{1` accessing 'exclude_lines.0'
2018-09-11T22:22:00.369Z	ERROR	[autodiscover]	cfgfile/list.go:104	Error creating runner from config: error parsing regexp: missing closing ): `GET\s"?/status|^\x1B\[([0-9]{1` accessing 'exclude_lines.0'

You'll notice that the regex for exclude_lines is getting clipped by filebeat 6.4.0, whereas filebeat 6.3.1 has no such issues.

Not sure why this is working fine in 6.3.1 and not 6.4.0, but something is obviously up.

Thanks,

Alex


(Carlos PĂ©rez Aradros) #2

Hi @Alex_Scoble,

Thank you for your feedback. This looks like a bug to me, as exclude_lines is a list, we take , as separators for different patterns, but that's wrong in this case, we should come with a way to avoid this issue.

Could you please open a new issue here https://github.com/elastic/beats/issues?

Best regards


(Alex Scoble) #3

Thanks, I just did that.

--Alex


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.