Logstash file plugin schedule

Elastic Stack Logstash
1

Hello,

How to cron task with logstash and file plugin ? with jdbc plugin it's integrate but not found with file plugin.

we use start_interval and discover_interval but not really work. ( stat_interval => 30
discover_interval => 30 )

How you do ?

rgds

2

How to cron task with logstash and file plugin ?

The file input plugin is designed to continuously monitor a set of files. Why would you want to stuff this in a crontab?

we use start_interval and discover_interval but not really work.

What did you try and in what way did it not work?

3

Hello Mr Baeck,

in the folder where logstash is looking for, we have a file ( *.log) we come in once per day with cron script. ( cron script del old file and build a new files) After that, we launch logstash in cron.

Why cron ? Because when we add stat_interval => 30 and discover_interval => 30 it don't seems working and we launch it because we have others index to run from differents plugins.

In fact, we are looking for a way to say to logstash :

  • when you see in the folder1 one new file, you index it according to conf files folder1.conf ( file come once per day )

-when you see new files ( plenty of files come every minutes) in the folder2, you index them according to conf files folder2.conf ( files come from every minutes )

4
  • when you see in the folder1 one new file, you index it according to conf files folder1.conf ( file come once per day )
  • when you see new files ( plenty of files come every minutes) in the folder2, you index them according to conf files folder2.conf ( files come from every minutes )

That's easy. Either use multiple pipelines or have a single pipeline and configure each file input to add a tag or a field that identifies each event, and use that tag or field to select which filters and outputs to use.

5

The best easy way was cron :wink:

Have you examples or link to ?

6

The best easy way was cron :wink:

I don't agree. There's no easy way of shutting down Logstash once the input files have been consumed.

Have you examples or link to ?