Logstash doesn't parse logs if i don't define start_position

Elastic Stack Logstash
#1

Below is my code. When I try to run this pipeline. Pipeline is starting but not parsing the logs. I have deleted .sincedb files at (data/plugins/inputs/file).

input {
    file {
        path => ".../directory/*.log"
    }
}
filter {
    
}
output { 
    stdout{
        codec => rubydebug
    }
}

But if I put,

start_position => "beginning"

That works. But everytime it starts from beginning, not where it left. If i put the below code and delete .sincedb files. It still doesn't parse.

start_position => "end"

In the first case that i didn't define start_position or define start_position => "end". That waits at "Successfully started Logstash API endpoint {:port=>9601}" But both cases I deleted .sincedb files. How can i make logstash continue where it left ? Thanks for answering

#2

Hi,

The default for start_position is end so it will only ingest entries that are written after LogStash found the file for the first time. Are you sure there were log entries written after that? Have you tried setting the log.level(logstash.yml | Logstash Reference [7.11] | Elastic) to debug or trace to find more information why the files are not read?

This should not happen - did you delete the sincedb file? The start_position is only used when detecting a new file. After that, the current position is stored in the sincedb file.

Best regards
Wolfram

#3

I think thats the answer.I have checked the box. I didn't get same problem again after your answer. If something could be different I will comment it. Thanks again.

#4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.