What I'm about to describe works fine in filebeat 6.3.1. The below only happens when I upgrade the filebeat daemonset for kubernetes to 6.4.0.
We are using hints based autodiscovery in kubernetes. The pod annotations are typically something like
annotations:
co.elastic.logs/exclude_lines: 'GET\s\"?/status|^\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)[m|K]\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)[m|K]$'
co.elastic.logs.service-foo/multiline.pattern: '^(\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K](\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K])?)?([0-9]{2}:[0-9]{2}|<[0-9]{2}>[0-9]\s[0-9]{4}-[0-9]{2}-[0-9]{2})'
co.elastic.logs.service-foo/multiline.negate: 'true'
co.elastic.logs.service-foo/multiline.match: 'after'
co.elastic.logs.service-foo-nginx/module: 'nginx'
co.elastic.logs.service-foo-nginx/fileset.stdout: 'access'
co.elastic.logs.service-foo-nginx/fileset.stderr: 'error'
The errors we are getting in filebeat 6.4.0 are like below:
2018-09-11T22:22:00.369Z ERROR fileset/factory.go:105 Error creating input: error parsing regexp: missing closing ): `GET\s"?/status|^\x1B\[([0-9]{1` accessing 'exclude_lines.0'
2018-09-11T22:22:00.369Z ERROR [autodiscover] cfgfile/list.go:104 Error creating runner from config: error parsing regexp: missing closing ): `GET\s"?/status|^\x1B\[([0-9]{1` accessing 'exclude_lines.0'
You'll notice that the regex for exclude_lines is getting clipped by filebeat 6.4.0, whereas filebeat 6.3.1 has no such issues.
Not sure why this is working fine in 6.3.1 and not 6.4.0, but something is obviously up.
Thanks,
Alex