Filebeat Version: 8.3.2 (image gets from docker.elastic.co/beats/filebeat:8.3.2)
I have roughly 40k logs in an NFS folder for Filebeat to collect. The Pod keeps restart itself in every ~10 minutes. And showing this error:
panic: reflect.Select: too many cases (max 65536)
goroutine 258 [running]:
reflect.Select({0xc0e5f7a000?, 0x10001?, 0x14124?})
/usr/local/go/src/reflect/value.go:2793 +0x79a
github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*Pipeline).runSignalPropagation(0xc0010827b8?)
/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/pipeline.go:329 +0x1d8
created by github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*Pipeline).registerSignalPropagation.func1
/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/pipeline.go:314 +0x96
I believe i have enough Memory, hundreds of GB. ulimit shows:
# ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 3095242
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1048576
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Here is my filebeat.yaml inputs:
- type: filestream
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /mnt/*
encoding: utf-8
close_inactive: 1m
tail_files: true
file_identity.path: ~
Any hint? Thanks
FYI, this is similar to this case