File input to grok filter to elasticsearch output

[cm47 (Conor M)]

Hi, I am trying to configure logstash to read lines from a custom file.

Logstash configuration file:
input {
file {
path => "/home/centos/customlog/testfile.log"
codec => line
start_position => "beginning"
ignore_older => 0
}
}

filter {
grok {
patterns_dir => "/etc/logstash/patterns"
match => { "message" => "%{DATETIME_PATTERN:timestamp} %{LOGLEVEL:log-level} %{GREEDYDATA:message}" }
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
user => "elastic"
password => "omitted"
index => "custom-log-test"
}
}

where DATETIME_PATTERN is a custom pattern declared like so:
DATETIME_PATTERN %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[ ]%{HOUR}:%{MINUTE}:%{SECOND}

Here is part of the log trying to be read:
2017-10-05 13:10:00 INFO Running Check Time issue
2017-10-05 13:10:00 INFO No need to alert - there are no future times
2017-10-05 13:20:00 INFO Running Check Time issue
2017-10-05 13:20:00 ERROR Issue found - sending alert

Where am I going wrong here? For testing purposes, I have also tried with stdout output using rubydebug codec and still don't see anything happening in logstash logs when I write and quit the file (logstash logs are raised to debug level). I also tried without the line codec incase it was not correct however I do believe that is the right codec to use. I also tried with a trivial grok pattern like just the log level and greedy data afterward and creating a log to match that pattern with no success as well. Any direction would be greatly appreciated, thanks!

[warkolm (Mark Walkom)]

It's probably a sincedb issue, so try setting that to /dev/null.

[cm47 (Conor M)]

Thanks for your reply, Mark. Shouldn't setting start position to beginning negate that issue? Anyway, I tried setting the sincedb path to /dev/null and no change. Still not seeing anything in logstash logs when I write and quit the test log file.

The only thing remotely related to this whole process that I see in the logs is this single line:

[DEBUG][logstash.inputs.file ] _globbed_files: /home/centos/customlog/testfile.log: glob is: []

Does this indicate something is wrong? Sorry, fairly inexperienced with ELK and just trying to figure out why I can't get something that should be very basic to work.

[cm47 (Conor M)]

I was able to resolve the issue. For whatever reason, logstash did not like the log file being in the centos user path that I had posted in my configuration. I changed it to /etc/logstash/customlog/testfile.log and I was able to see lines from the file printed to stdout.