Heavy Load due file beat service


(Kunalpatil3017) #1

Hello Im using Filebeat service in my graylog server (Graylog-Sidecar collector)
But Due to some reason it is consuming too much resources on my linux server
load is not due to data read it is due to filebeat debug mode


(Magnus Bäck) #2

Any clues in the Filebeat log file?


(Kunalpatil3017) #3

No error so far


(Magnus Bäck) #4

It's not only errors that can be interesting here. The CPU usage indicates that Filebeat is stuck in a tight loop, and the log could contain clues about what it's doing.


(Kunalpatil3017) #5

Debug mode still no error


(Kunalpatil3017) #6

Now once i run above command server load doubled


(Kunalpatil3017) #7

How to identify the issue I'm stuck here

Can u send me some debugging steps so i will able to find out the problem


(Magnus Bäck) #8

Please don't post screenshots. Use copy/paste.

I don't know what log file that is or what graylog-collector-sidecar does. Please show actual Filebeat log entries, typically found in the Filebeat log file.


(Kunalpatil3017) #9
filebeat:
  prospectors:
  - document_type: log
    fields:
      gl2_source_collector: f43bb48d-9343-40e9-bc83-6936e8e5bb61
    ignore_older: 10s
    input_type: log
    paths:
    - /var/log/httpd/adminbetanv1.justbuylive.in-access_log
    scan_frequency: 0
    tail_files: false
output:
  logstash:
    hosts:

(Kunalpatil3017) #10

server_url: http://graylog2.justbuylive.in/api/
update_interval: 10
tls_skip_verify: false
send_status: true
list_log_files:
- /var/log/httpd/
node_id: graylog-collector-sidecar
collector_id: file:/etc/graylog/collector-sidecar/collector-id
log_path: /var/log/graylog/collector-sidecar
log_rotation_time: 86400
log_max_age: 604800
tags:

- apache

backends:
- name: filebeat
enabled: true
binary_path: /usr/bin/filebeat
configuration_path: /etc/graylog/collector-sidecar/generated/filebeat.yml


(Magnus Bäck) #11

I asked for the logs, not the configuration. Anyway, it seems you're actually running Graylog Collector Sidecar, which includes a patched Filebeat? I have no idea how that works. Maybe someone else can help.


(Tudor Golubenco) #12

Also don't know much about the graylog setup, but:

This could be the issue. Try increasing that to at least 1s.


(Kunalpatil3017) #13

Thanks tudor
that was correct suggestion due to that parameter only load had increased


(system) #14

This topic was automatically closed after 21 days. New replies are no longer allowed.