I'm seeing a bunch of errors when configuring filebeat with multiple filestream
input types.
The configuration is simple enough, filebeat (7.15.1) running on Ubuntu 18.04 LTS forwards everything to logstash using beats.
There are a few inputs defined, below is an example. With the exclude_lines
entry commented out, filebeat starts and runs. Uncomment the exclude_lines directive and it crashes with a panic at
panic: unlock on unlocked mutex
goroutine 106 [running]:
github.com/elastic/go-concert/unison.Mutex.Unlock(0xc000aaa7e0)
/go/pkg/mod/github.com/elastic/go-concert@v0.2.0/unison/mutex.go:132 +0x65
github.com/elastic/beats/v7/filebeat/input/filestream/internal/input-logfile.releaseResource(0xc000c760b0)
/go/src/github.com/elastic/beats/filebeat/input/filestream/internal/input-logfile/harvester.go:296 +0x31
github.com/elastic/beats/v7/filebeat/input/filestream/internal/input-logfile.(*defaultHarvesterGroup).Continue.func1(0x555e16f4a7d8, 0xc000747240, 0x0, 0x0)
/go/src/github.com/elastic/beats/filebeat/input/filestream/internal/input-logfile/harvester.go:246 +0x37d
github.com/elastic/go-concert/unison.(*TaskGroup).Go.func1(0xc00042a3c8, 0xc0000ab310)
/go/pkg/mod/github.com/elastic/go-concert@v0.2.0/unison/taskgroup.go:163 +0xb1
created by github.com/elastic/go-concert/unison.(*TaskGroup).Go
/go/pkg/mod/github.com/elastic/go-concert@v0.2.0/unison/taskgroup.go:159 +0xdc
filebeat.inputs:
# Template: fail2ban
- type: filestream
paths:
- "/var/log/fail2ban.log*"
prospector.scanner.exclude_files:
- '\.gz$'
rotation.external.strategy.copytruncate:
suffix_regex: '\.\d+$'
fields:
application: fail2ban
# # Generic to all nodes
# Template: ufw
- type: filestream
paths:
- "/var/log/ufw.log*"
prospector.scanner.exclude_files:
- '\.gz$'
rotation.external.strategy.copytruncate:
suffix_regex: '\.\d+$'
fields:
application: ufw
# Template: unattended-upgrades
- type: filestream
paths:
- "/var/log/unattended-upgrades/unattended-upgrades-dpkg.log*"
prospector.scanner.exclude_files:
- '\.gz$'
rotation.external.strategy.copytruncate:
suffix_regex: '\.\d+$'
# exclude_lines:
# - '^\s*$'
parsers:
- multiline:
type: pattern
pattern: '^Log started:\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}$'
negate: true
match: after
fields:
application: "unattended-upgrades"
part: "dpkg-output"
Another simple configuration such as:
filebeat.inputs:
# Template: fail2ban
- type: filestream
paths:
- "/var/log/fail2ban.log*"
prospector.scanner.exclude_files:
- '\.gz$'
rotation.external.strategy.copytruncate:
suffix_regex: '\.\d+$'
fields:
application: fail2ban
# # Generic to all nodes
# Template: ufw
- type: filestream
paths:
- "/var/log/ufw.log*"
prospector.scanner.exclude_files:
- '\.gz$'
rotation.external.strategy.copytruncate:
suffix_regex: '\.\d+$'
fields:
application: ufw
# Template: ubuntu-advantage
- type: filestream
paths:
- "/var/log/ubuntu-advantage.log*"
prospector.scanner.exclude_files:
- '\.gz$'
rotation.external.strategy.copytruncate:
suffix_regex: '\.\d+$'
fields:
application: ubuntu-advantage
also crashes with the same panic, there's nothing complicated present there in the configurations.
Any thoughts or solutions?