Logstash file read basic question

Elastic Stack Logstash
1

please point me in the right direction. I just started with logstash. Thanks

My config file
cat test.conf
input { file { path => "/localhome/user/test.txt" start_position => "beginning" } } filter { grok { match => { "message" => "%{NUMBER:age},%{WORD:name}" } } } output { file { path => "/localhome/user/outfile.txt" } }
started logstash as
logstash -f test.conf
Logstash startup completed

Now i created a test file as below
cat test.txt
25,Mark
23,Sammy
24,Luna

save the file, and i am expecting to see a new file output.txt with three records
but this is what i got
cat outfile.txt
{"message":"25,Mark","@version":"1","@timestamp":"2016-03-01T05:40:41.172Z","path":"/localhome/user/test.txt","host":"hostname","age":"25","name":"Mark"}

why is it reading only the first record?

Now i edited the file test.txt and added one more record
now this is what i see in output.txt

]cat outfile.txt
{"message":"25,Mark","@version":"1","@timestamp":"2016-03-01T05:40:41.172Z","path":"/localhome/user/test.txt","host":"hostname","age":"25","name":"Mark"}
{"message":"23,Sammy","@version":"1","@timestamp":"2016-03-01T05:40:41.176Z","path":"/localhome/user/test.txt","host":"hostname","age":"23","name":"Sammy"}
{"message":"24,Luna","@version":"1","@timestamp":"2016-03-01T05:40:41.178Z","path":"/localhome/user/test.txt","host":"hostname","age":"24","name":"Luna"}
{"message":"25,Mark","@version":"1","@timestamp":"2016-03-01T05:42:58.362Z","path":"/localhome/user/test.txt","host":"hostname","age":"25","name":"Mark"}

didn't quite understand this behavior, why is it not reading the whole file? why is it missing the last record and also repeating the first record?

2

You need to set sincedb_path to something like /dev/null.

Take a read of https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb_path

3

yes, i cleared $HOME/.sincedb* before starting logstash

with no .sincedb files, with no data file, i started logstash
and then created a new data file. then i expect it to auto read the entire file form the beginning and write it to output file, but in output file i only see 1 line in first pass.