Filebeat filestream input pause when disk queue full?

Elastic Stack Beats
1

Dear,

I tried searching for an answer, but didn't immediately find one.

I'm using filebeat's filestream input (works great!) and setup the disk queue to a max_size.

The documentation of the internal queue states the following:
"Events that exceed this maximum will either pause their input or be discarded, depending on the input's configuration".
I can't really find the setting on the filestream input that explicitly sets the behaviour of pauzing the input instead of discarding messages.
What is the setting to make sure filestream pauzes when the disk queue is full?
Or is this the default behaviour?

Many thanks in advance.

2

I don't think there is a setting, this depends on your input, for some inputs filebeat will pause, for other inputs it will discard the messages.

For filestream inputs when the disk queue is full filebeat will stop reading new events from the file and will continue from the last read position when there is space in the disk queue, but keep in mind that if the file is rotated while filebeat is not reading it, you may lost some messages.

For TCP/UDP inputs, filebeat will drop new events, which will them be lost.

3

Dear,

Many thanks for your quick and clear answer.
This makes sense. :slight_smile:

Thank you very much.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.