Filebeat creating multiple threads and consuming system resources

Elastic Stack Beats
1

I have enable the filebeat, it started with consuming systems resources and creating multiple threads.
I installed filebeat on a lunix server and ‘yml’ configuration file is as below. Would you please look into the same and support.

I am using filebeat 6.4.2

###################### Filebeat Configuration #########################

  • type: log
    enabled: true
    paths:
    - /applog/logs/log_MS202.log*

    multiline.pattern: ^####<
    multiline.negate: true
    multiline.match: after

    fields:
    index_suffix: prod
    env: prod
    fields_under_root: true
    harvester_limit: 1
    close_inactive: 1m
    close_eof: true
    #============================= Filebeat modules ===============================

filebeat.config.modules:

Glob pattern for configuration loading

path: ${path.config}/modules.d/*.yml

Set to true to enable config reloading

reload.enabled: false

#================================ General =====================================

#----------------------------- Logstash output --------------------------------
output.logstash:

The Logstash hosts

hosts: ["p1osbansr01.btc.com.bh:5044"]

compression_level: 3
max_retries: -1

#================================ Logging =====================================

Sets log level. The default log level is info.

Available log levels are: error, warning, info, debug

logging.level: info

At debug level, you can selectively enable logging only for some components.

To enable all selectors use ["*"]. Examples of other selectors are "beat",

"publish", "service".

logging.selectors: ["*"]

logging.to_files: true

logging.files:
path: /var/log/filebeat
rotateeverybytes: 10485760 # = 10MB
keepfiles: 7

filebeat 24780 24779 0 10:10 ? 00:00:17 /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat -d * -c /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat.yml
filebeat 25190 25180 0 10:15 ? 00:00:00 /bin/sh -c /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeatstart.sh
filebeat 25191 25190 0 10:15 ? 00:00:15 /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat -d * -c /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat.yml
filebeat 26722 26706 0 10:20 ? 00:00:00 /bin/sh -c /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeatstart.sh
filebeat 26723 26722 0 10:20 ? 00:00:11 /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat -d * -c /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat.yml
filebeat 27507 27498 0 10:25 ? 00:00:00 /bin/sh -c /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeatstart.sh
filebeat 27508 27507 0 10:25 ? 00:00:10 /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat -d * -c /home/filebeat/filebeat-6.4.2-linux-x86_64/filebeat.yml
filebeat 28058 28046 0 10:30 ? 00:00:00 /bin/sh -c /home/filebeat/filebeat-6.4.2-linux-x86

2

Could you please format the config file using </>? Also, please share the debug logs of Filebeat (./filebeat -e -d "*").

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.