Logstash file-input problem

[sivap (sivakumar)]

Hi Team,

Every 5 minutes once my cron job will create one new text file ( filename with time - jobs_00:00.txt, jobs_00:05.txt, jobs_00:10.txt, jobs_00:15.txt, and so on..). So I have created logstash config using file-input to read the content from those file in the specified location (/mnt/arlo_input/*).

But here some point of time logstash is not reading the log files. Any ideas what could be the problem?

Please find the logstash config as below,
input {
file {
path => "/mnt/arlo_input/*"
start_position => "beginning"
sincedb_path => "/dev/null"
ignore_older => 0
discover_interval => 2
}
}

Please, could someone help me!

Thanks
Siva.P

[Badger]

@sivap

path => "/mnt/arlo_input/*"

Is that a network mount? If so, did you read the caveat in the documentation?

@sivap

ignore_older => 0

You are telling it to ignore any file that was modified more than zero seconds ago. Which is pretty much every file, right? What are you trying to achieve using this option?

[Charaf_Ahmed]

Simple :

input{
        path => "/mnt/arlo_input/*.txt"
   }

[sivap (sivakumar)]

@Badger & @Charaf_Ahmed Thanks for the response!

I have removed the ignore_older parameter and tried *.txt. But still, logstash missing some files to read.

Could you please suggest some solution.

[Badger]

Enable debug logging. Pay close attention to the _globbed_files and _discover_file lines in the resulting output.

[Charaf_Ahmed]

Remove everything and just leave the path to make sure everything is OK. Then make another change