File input doesn t read incoming Json file from web application


#1

Hi everyone,

I'm trying to parse incoming json files from my web app stored in folder /tmp/test/*.json.

The problem is that logstash doesnt read my files when there are created. I have to modify them so that logstash parses them.

I try to change parameters in my input file like ignore_older to parse all logs but changed nothing

I did this filter.conf:

input {
file {
path => "/tmp/test/*.json"
codec => json
start_position => "beginning"
ignore_older => 0
}
}

filter {
json {
source => "message"
}
}

output {
stdout {codec => json}
elasticsearch {
hosts => "127.0.0.1:9200"
index => "test-%{+YYYY.MM}"
codec => json
}
}

Below, this is my folder with files permissions

I started logstash with --debug and -v but there is no error message. The .sincedbpath is not updated when the log is created.


#2

This says to ignore all files older than 0 seconds. Which means it ignores all files.


#3

Thanks for your reply !

I have removed the ignore_older parameter from my config. Now logstash see my file and add an entry into .sincedbpath. However, The current byte offset within the file is 0 so I understand that logstash doesnt read the file ?

Sincedb information:
262169 0 2049 500 1552734148.288747 -> file parsed by logstash if I do a modification
262195 0 2049 0 1552734148.2914188 -> file incoming not parsed by logstash.

Still no error or info message on logs of logstash.
If you need more information, ask me.


#4

Run with '--log.level trace' to see what filewatch is doing.