# Heavy Load due file beat service

**URL:** https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571
**Category:** Beats
**Tags:** filebeat
**Created:** [October 21, 2016, 6:44am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571 "2016-10-21T06:44:12Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [October 21, 2016, 6:44am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/1 "2016-10-21T06:44:12Z")

</div>

Hello Im using Filebeat service in my graylog server (Graylog-Sidecar collector)  
But Due to some reason it is consuming too much resources on my linux server  
load is not due to data read it is due to filebeat debug mode

 ![](https://us1.discourse-cdn.com/elastic/original/2X/4/42f108a7e998f91ecd54dc79e86cbd1f69a1c0f2.PNG)

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 21, 2016, 7:09am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/2 "2016-10-21T07:09:50Z")

</div>

Any clues in the Filebeat log file?

---

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [October 21, 2016, 7:14am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/3 "2016-10-21T07:14:22Z")

</div>

No error so far

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 21, 2016, 7:21am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/4 "2016-10-21T07:21:54Z")

</div>

It's not only errors that can be interesting here. The CPU usage indicates that Filebeat is stuck in a tight loop, and the log could contain clues about what it's doing.

---

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [October 21, 2016, 7:27am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/5 "2016-10-21T07:27:34Z")

</div>

Debug mode still no error

 ![](https://us1.discourse-cdn.com/elastic/original/2X/0/0bbbd729686b79b925693d8fbd5ad37ba70bf42f.PNG)

---

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [October 21, 2016, 7:30am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/6 "2016-10-21T07:30:12Z")

</div>

Now once i run above command server load doubled

 ![](https://us1.discourse-cdn.com/elastic/original/2X/d/d1eb6fa35dfcf57bb2fbe035dd479bb941ddf992.PNG)

---

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [October 21, 2016, 7:50am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/7 "2016-10-21T07:50:13Z")

</div>

How to identify the issue I'm stuck here

Can u send me some debugging steps so i will able to find out the problem

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 21, 2016, 7:51am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/8 "2016-10-21T07:51:19Z")

</div>

Please don't post screenshots. Use copy/paste.

I don't know what log file that is or what graylog-collector-sidecar does. Please show actual Filebeat log entries, typically found in the Filebeat log file.

---

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [October 21, 2016, 10:33am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/9 "2016-10-21T10:33:29Z")

</div>

```auto
filebeat:
  prospectors:
  - document_type: log
    fields:
      gl2_source_collector: f43bb48d-9343-40e9-bc83-6936e8e5bb61
    ignore_older: 10s
    input_type: log
    paths:
    - /var/log/httpd/adminbetanv1.justbuylive.in-access_log
    scan_frequency: 0
    tail_files: false
output:
  logstash:
    hosts:

```

---

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [October 21, 2016, 10:35am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/10 "2016-10-21T10:35:17Z")

</div>

server\_url: [http://graylog2.justbuylive.in/api/](http://graylog2.justbuylive.in/api/)  
update\_interval: 10  
tls\_skip\_verify: false  
send\_status: true  
list\_log\_files:  
- /var/log/httpd/  
node\_id: graylog-collector-sidecar  
collector\_id: file:/etc/graylog/collector-sidecar/collector-id  
log\_path: /var/log/graylog/collector-sidecar  
log\_rotation\_time: 86400  
log\_max\_age: 604800  
tags:

```
- apache

```

backends:  
- name: filebeat  
enabled: true  
binary\_path: /usr/bin/filebeat  
configuration\_path: /etc/graylog/collector-sidecar/generated/filebeat.yml

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 21, 2016, 10:55am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/11 "2016-10-21T10:55:41Z")

</div>

I asked for the logs, not the configuration. Anyway, it seems you're actually running Graylog Collector Sidecar, which includes a patched Filebeat? I have no idea how that works. Maybe someone else can help.

---

<div class="post-metadata">

### Author: ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [October 21, 2016, 10:59am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/12 "2016-10-21T10:59:10Z")

</div>

Also don't know much about the graylog setup, but:

> [@kunalpatil3017](#):
>
> scan\_frequency: 0

This could be the issue. Try increasing that to at least `1s`.

---

<div class="post-metadata">

### Author: ![kunalpatil3017](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kunalpatil3017/32/12632_2.png) [@kunalpatil3017](https://discuss.elastic.co/u/kunalpatil3017)
#### Post date: [November 9, 2016, 2:32pm UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/13 "2016-11-09T14:32:02Z")

</div>

Thanks tudor  
that was correct suggestion due to that parameter only load had increased

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 11, 2016, 6:44am UTC](https://discuss.elastic.co/t/heavy-load-due-file-beat-service/63571/14 "2016-11-11T06:44:30Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
