# What configs exist for lower latency reads?

**URL:** <https://discuss.elastic.co/t/what-configs-exist-for-lower-latency-reads/168295>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 13, 2019, 9:27pm UTC](https://discuss.elastic.co/t/what-configs-exist-for-lower-latency-reads/168295 "2019-02-13T21:27:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [February 13, 2019, 9:27pm UTC](https://discuss.elastic.co/t/what-configs-exist-for-lower-latency-reads/168295/1 "2019-02-13T21:27:18Z")

</div>

Hi, what are the config settings for filebeat to scan lines as frequently as possible.

So I will have two prospectors.  
1- One that reads logs regularly, using default settings  
2- One that reads "events" (separate file) as frequently as possible.

I understand there is scan\_frequency but this is how often new files are checked...

What params exist to lower the ingestion and delivery of new lines? And yes I understand this can be CPU intensive, so I will tune for my scenario...

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 20, 2019, 4:27pm UTC](https://discuss.elastic.co/t/what-configs-exist-for-lower-latency-reads/168295/2 "2019-02-20T16:27:49Z")

</div>

Filebeat tries to read until EOF as fast as possible. Well, subject to backpressure from the memory queue and outputs. Once EOF is reached it backs off before trying to read more lines. See [`backoffX` settings](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#_literal_backoff_literal).

The event queue buffers events. By default the queue is flushed once full or 1s after the first event filling the queue is received. Setting the flush timeout to 0 can improve latency if you only have a small amount of events. See [Internal queue docs](https://www.elastic.co/guide/en/beats/filebeat/current/configuring-internal-queue.html).  
With a timeout of 0 the queue implementation tries to forward events immediately, but still can buffer up batches in case outputs are busy.

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [February 21, 2019, 4:09pm UTC](https://discuss.elastic.co/t/what-configs-exist-for-lower-latency-reads/168295/3 "2019-02-21T16:09:49Z")

</div>

Ok cool my output is actually Kafka. I'll try it out...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 21, 2019, 4:09pm UTC](https://discuss.elastic.co/t/what-configs-exist-for-lower-latency-reads/168295/4 "2019-03-21T16:09:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
