# Filebeat settings

**URL:** <https://discuss.elastic.co/t/filebeat-settings/323845>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 24, 2023, 2:58pm UTC](https://discuss.elastic.co/t/filebeat-settings/323845 "2023-01-24T14:58:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![alex\_96](https://avatars.discourse-cdn.com/v4/letter/a/ec9cab/32.png) [@alex\_96](https://discuss.elastic.co/u/alex_96)\
**Post date:** [January 24, 2023, 2:58pm UTC](https://discuss.elastic.co/t/filebeat-settings/323845/1 "2023-01-24T14:58:53Z")

</div>

Hello, I want to clarify if I understood the documentation correctly,  
` bulk_max_size` - filebeat transmits events (as I understand it, let's say a line in the log file) bundled, default 50, if I set 0, then `queue` comes into play,  
`queue events` - this is also the number of events (max number of events) that are also collected in packets and sent to the output, there is also `flush.min_events` (the minimum number of events to send) and `flush.timeout` (the maximum amount of time that the event waits to send)  
Did I understand correctly that if `bulk_max_size`, `queue events`, `flush.min_events` and `flush.timeout` are set to `0`, then there is no event queue and they are sent in real time?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [January 25, 2023, 3:23am UTC](https://discuss.elastic.co/t/filebeat-settings/323845/2 "2023-01-25T03:23:42Z")

</div>

Hi @alex_96

Yes looks like you basically have it correct.

If you set `max_bulk_size` to `0` the `queue.mem` settings determine the behavior.

I would ask you what you are trying to accomplish... I can tell you that that sense to set everything to `0` so that everything is "Realtime" is not a "best practice" and with any volume will actually have the opposite effect, most likely slowing down the throughput.

The defaults are pretty good if you have any scale and then typically adjust them.

Not to say you can not adjust, test try, etc... I encourage you to do that, but at scale, we are rarely if ever setting those to 0 as elasticsearch and filebeat are optimized to send bulk batches.

If you are sending to a scaled cluster increasing the workers can have a positive affect.

---

<div class="post-metadata">

**Author:** ![alex\_96](https://avatars.discourse-cdn.com/v4/letter/a/ec9cab/32.png) [@alex\_96](https://discuss.elastic.co/u/alex_96)\
**Post date:** [January 25, 2023, 8:37am UTC](https://discuss.elastic.co/t/filebeat-settings/323845/3 "2023-01-25T08:37:27Z")

</div>

@stephenb Hello,  
Thanks for the answer, it's just that I've been trying to figure out for several weeks why events arrive with a delay of a day  
I will briefly describe the problem, I have 2 hosts, one with elasticsearch 7.17.5, the other with filebeat 7.17.8, ids(suricata, zeek) are installed on the host with filebeat  
zeek events arrive in real time, while suricata events are ~day behind  
Example:

```auto
event.original - {"timestamp":"2023-01-23T21:11:00.550666+0000
event.ingested - Jan 25, 2023 @ 11:22:47.811
@timestamp - Jan 24, 2023 @ 00:11:00.550

```

Everything is fine with the `@timestamp`, but I still can’t understand why the events reach a day later  
Filebeat config:

```auto
scan_frequency: 10s
close_inactive: 15m
output.elasticsearch:
  hosts: ["https://192.168.200.20:9200"]
  username: "username"
  password: "password"
  ssl.verification_mode: "none"
  indices:
    - index: "filebeat-7.17.8-suricata-%{+yyyy.ww}"
      when.equals:
       event.module: "suricata"
    - index: "filebeat-7.17.8-zeek-%{+yyyy.ww}"
      when.equals:
       event.module: "zeek"
    - index: filebeat-7.17.8-threatintel-%{+yyyy.ww}
      when.equals:
       event.module: "threatintel"
setup.template.name: "filebeat-7.17.8"
setup.template.pattern: "filebeat-7.17.8-*"
setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 0
bulk_max_size: 0
worker: 4
#compression_level: 9
queue.mem:
  flush.min_events: 0
  flush.timeout: 0s

```

Filebeat host has 4 cores(with 1 thread per core) 8 ram, 2tb disk space  
If you need anything else, I'll be happy to answer.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 22, 2023, 10:38am UTC](https://discuss.elastic.co/t/filebeat-settings/323845/4 "2023-02-22T10:38:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.