Close_older not working as expected

Elastic Stack Logstash
1

I'm running logstash from a bash script. I have the following input section :
input {
file {
path => "/home/pxg110/logstash/mach/test.csv"
start_position => "end"
sincedb_path => "/dev/null"
close_older => 1
}
}
I want logstash to time out after 1 second according to the online docs shown below.
What do I need to do to get this to time out properly.

The file input closes any files that were last read the specified timespan in seconds ago. This has different implications depending on if a file is being tailed or read. If tailing, and there is a large time gap in incoming data the file can be closed (allowing other files to be opened) but will be queued for reopening when new data is detected. If reading, the file will be closed after closed_older seconds from when the last bytes were read. The default is 1 hour

2

Why are you disabling the sincedb functionality? I am not sure what you are looking to achieve with this configuration - could you please elaborate a bit more?

3

I want logstash to time out after 1 second according to the online docs shown below.

What do you mean by time out? Shut down if the file hasn't seen any activity for more than one second? That's not what close_older does.

4

The behaviour that I need is for logstash to exit once it has processed all bytes reading from the beginning to the end the file as specified by the file . As logstash is invoked from a bash script it cannot wait until more data comes in.

From the documentation I read and copied in my submission, when you set it to read from the beginning of the file, it will exit after the number of seconds specified has elapsed.

If this not possible, I would need to have logstash monitor a folder for changing content and have it process the file once it its submitted.

If you have examples, that would be great.
PS: the /dev/null was used in an example to disable the creation of a sincdb file that keeps track where logstash is while processing a file. This si used in case of failure so it knows where to restart.

5

You could make Logstash stop once the file has been processed if you pipe it into a stdin input plugin rather than using the file input plugin.

6

Christian
I searched and found a post by Marcus suggesting the use of the stdin plugin and piping data into logstash. Just tried it and it works like I need it to work.

Thanks all for your help

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.