I changed filebeat.input to filestream id and enabled both modules. Still the same behavior.
Every 2.0s: systemctl status filebeat HOSTls: Thu Mar 7 15:35:10 2024
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2024-03-07 15:27:31 CET; 7min ago
Docs: https://www.elastic.co/beats/filebeat
Main PID: 322416 (filebeat)
Tasks: 8 (limit: 74760)
Memory: 45.9M (max: 6.0G)
CGroup: /system.slice/filebeat.service
└─322416 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat
Mar 07 15:27:31 HOSTols systemd[1]: Started Filebeat sends log files to Logstash or directly to Elasticsearch..
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
512.08kB 2.89% 88.42% 512.08kB 2.89% regexp.compile
(pprof) top
Showing nodes accounting for 15643.65kB, 88.42% of 17691.72kB total
Showing top 10 nodes out of 60
flat flat% sum% cum cum%
5488.12kB 31.02% 31.02% 5488.12kB 31.02% github.com/goccy/go-json/internal/decoder.init.0
3786.34kB 21.40% 52.42% 3786.34kB 21.40% github.com/elastic/beats/v7/libbeat/asset.GetFields
2756.97kB 15.58% 68.01% 2756.97kB 15.58% github.com/goccy/go-json/internal/encoder.init.0
522.06kB 2.95% 70.96% 522.06kB 2.95% cloud.google.com/go/pubsub/apiv1/pubsubpb.init
522.06kB 2.95% 73.91% 522.06kB 2.95% github.com/googleapis/gnostic/openapiv2.init
517.33kB 2.92% 76.83% 517.33kB 2.92% regexp/syntax.(*compiler).inst
513.31kB 2.90% 79.73% 513.31kB 2.90% google.golang.org/protobuf/internal/filedesc.(*File).initDecls
512.88kB 2.90% 82.63% 512.88kB 2.90% google.golang.org/protobuf/internal/strs.(*Builder).grow
512.50kB 2.90% 85.53% 512.50kB 2.90% runtime.allocm
512.08kB 2.89% 88.42% 512.08kB 2.89% regexp.compile
(pprof)
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 10ms, 100% of 10ms total
Showing top 10 nodes out of 22
flat flat% sum% cum cum%
10ms 100% 100% 10ms 100% runtime/internal/syscall.Syscall6
0 0% 100% 10ms 100% github.com/elastic/beats/v7/libbeat/api.makeAPIHandler.func1
0 0% 100% 10ms 100% github.com/elastic/elastic-agent-libs/monitoring.(*Func).Visit
0 0% 100% 10ms 100% github.com/elastic/elastic-agent-libs/monitoring.(*Registry).Visit (inline)
0 0% 100% 10ms 100% github.com/elastic/elastic-agent-libs/monitoring.(*Registry).doVisit
0 0% 100% 10ms 100% github.com/elastic/elastic-agent-libs/monitoring.CollectStructSnapshot
0 0% 100% 10ms 100% github.com/elastic/elastic-agent-system-metrics/metric/system/host.ReportInfo.func1
0 0% 100% 10ms 100% github.com/elastic/go-sysinfo.Host
0 0% 100% 10ms 100% github.com/elastic/go-sysinfo/providers/linux.(*reader).network
0 0% 100% 10ms 100% github.com/elastic/go-sysinfo/providers/linux.linuxSystem.Host
HEAP a second before MEM reaching 6 GB CAP and reseting
very 2.0s: systemctl status filebeat HOSTNAME: Thu Mar 7 15:55:55 2024
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2024-03-07 15:55:54 CET; 1s ago
Docs: https://www.elastic.co/beats/filebeat
Main PID: 331174 (filebeat)
Tasks: 8 (limit: 74760)
Memory: 121.4M (max: 6.0G)
CGroup: /system.slice/filebeat.service
└─331174 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat
Mar 07 15:55:54 HOSTNAME systemd[1]: filebeat.service: Service RestartSec=100ms expired, scheduling restart.
Mar 07 15:55:54 HOSTNAME systemd[1]: filebeat.service: Scheduled restart job, restart counter is at 7.
Mar 07 15:55:54 HOSTNAME systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Mar 07 15:55:54 HOSTNAME systemd[1]: Started Filebeat sends log files to Logstash or directly to Elasticsearch..
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
.bash_history .cache/ elks-kibana-hearbteat.pub pprof/ .ssh/
.bash_logout .config/ .local/ .python_history teraz-do-pobrania-heartbeat/
.bash_profile eclipse-jee-neon-3-win32-x86_64.zip logstash/ scripts/
[HOSTNAME]~$ watch sudo tail /var/log/filebeat/filebeat-20240307-236.ndjson
alerts/ .bashrc elks-kibana-hearbteat metricbeat-8.10.2-x86_64.rpm sftp/
.bash_history .cache/ elks-kibana-hearbteat.pub pprof/ .ssh/
.bash_logout .config/ .local/ .python_history teraz-do-pobrania-heartbeat/
.bash_profile eclipse-jee-neon-3-win32-x86_64.zip logstash/ scripts/
[HOSTNAME]~$ watch sudo tail /var/log/filebeat/filebeat-20240307-2
filebeat-20240307-234.ndjson filebeat-20240307-235.ndjson filebeat-20240307-236.ndjson filebeat-20240307-237.ndjson filebeat-20240307-238.ndjson filebeat-20240307-239.ndjson filebeat-20240307-240.ndjson filebeat-20240307-241.ndjson
[HOSTNAME]~$ watch sudo tail /var/log/filebeat/filebeat-20240307-2
filebeat-20240307-234.ndjson filebeat-20240307-235.ndjson filebeat-20240307-236.ndjson filebeat-20240307-237.ndjson filebeat-20240307-238.ndjson filebeat-20240307-239.ndjson filebeat-20240307-240.ndjson filebeat-20240307-241.ndjson
[HOSTNAME]~$ watch sudo tail /var/log/filebeat/filebeat-20240307-241.ndjson
[HOSTNAME]~$ sudo vi /etc/filebeat/filebeat.yml
[HOSTNAME]~$ sudo systemctl restart filebeat
[HOSTNAME]~$
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Type: inuse_space
Time: Mar 7, 2024 at 3:55pm (CET)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 3.66GB, 97.97% of 3.73GB total
Dropped 323 nodes (cum <= 0.02GB)
flat flat% sum% cum cum%
3.66GB 97.97% 97.97% 3.66GB 97.97% github.com/elastic/beats/v7/filebeat/inputsource/common/dgram.DatagramReaderFactory.func1.1
0 0% 97.97% 0.03GB 0.76% github.com/elastic/beats/v7/filebeat/beater.(*countingClient).Publish
0 0% 97.97% 3.66GB 97.97% github.com/elastic/beats/v7/filebeat/inputsource/common/dgram.(*Listener).Start.func1
0 0% 97.97% 3.66GB 97.97% github.com/elastic/beats/v7/filebeat/inputsource/common/dgram.(*Listener).connectAndRun
0 0% 97.97% 0.03GB 0.76% github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*client).Publish
0 0% 97.97% 0.03GB 0.76% github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*client).publish
0 0% 97.97% 0.03GB 0.76% github.com/elastic/beats/v7/libbeat/publisher/processing.(*group).Run
0 0% 97.97% 3.66GB 97.99% github.com/elastic/go-concert/unison.(*TaskGroup).Go.func1