We are currently using filebeat on a lot of servers, Linux Red Hat 7.2 or Windows 2008 server.
We have no issues with linux servers and we have a very low CPU consuption.
We have some troubles with filebeat 5.5.2 on Windows 2008 servers.
When we have high activity on a Windows server (1.6 GB/h), the CPU usage of filebeat is very high (> 15%) and it affects significantly our performance mesures.
Looking in code, il seems that buffered I/O are used even for heavily written log files.
Can we suspect a performance hole for this specific case ?
Does it exist a tool to monitor specifically the filebeat process on windows ?
My filebeat.yml (5.5.2)
#=========================== Filebeat prospectors ================================
filebeat.prospectors:
- input_type: log
paths:
- E:/EquiX/ate_perf.log
scan_frequency: 10s
harvester_buffer_size: 16384
max_bytes: 32766
tail_files: false
backoff: 1s
max_backoff: 10s
backoff_factor: 2
ignore_older: 30m
close_inactive: 5m
close_renamed: false
close_removed: true
close_eof: false
close_timeout: 0
encoding: windows-1252
fields:
type_log: CYCATEPERF_PRF
fields_under_root: true
enabled: true
#========================= Filebeat global options ============================
filebeat.spool_size: 2048
filebeat.idle_timeout: 5s
filebeat.registry_file: "C:/ProgramData/filebeat/registry"
#filebeat.config_dir: "C:/Program Files/Filebeat/config"
filebeat.shutdown_timeout: 0
#================================ Outputs =====================================
#------------------------------- Kafka output ----------------------------------
output.kafka:
enabled: true
hosts: ["UD0TX046:9092","UD0TX047:9092","UD0TX048:9092"]
topic: 'CYCATEPERF_PRF'
version: 0.9.0.1
worker: 1
compression: gzip
#================================ Logging ======================================
logging.level: info
logging.to_files: true
logging.files:
path: "E:/filebeat/Logs"
name: filebeat.log
rotateeverybytes: 1485760
keepfiles: 10