Last line of log not picked up by logstash

[Vivek_Sharma]

Hi

I am using logstash version 5 and trying to parse a log file.

When there are 10 records in the log file, logstash processes only 9. Not picking up the last line of the log.

I picked the last record and tried to parse it on test grok patterns and it worked fine.

I also copy pasted the record again in the log and it was successfully picked up by the logstash.

But I am not able to solve out the issue that if we have multiple lines of log then why logstash is not picking up the last line.

Here is my logstash configuration file:

input {

beats {
port => 5044
}
}

filter {
grok {
match => { "message" => ["iDoc_nbr:%{USERNAME:idoc_number}|status:%{GREEDYDATA:log_message}|error_code:%{USERNAME:error_code}|error_msg:%{GREEDYDATA:error_message}|error_desc:%{GREEDYDATA:error_description}"
] }
}

mutate {
  add_field => { "tech_stack" => "%{[fields][tech_stack]}" }
  add_field => { "kpi_type" => "%{[fields][kpi_type]}" }
}

}

output {
elasticsearch {
hosts => "localhost:9200"
user => "elastic"
password => "changeme"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
stdout { codec => rubydebug }
}

Please help.

[abhiroyg]

Please have a look at this:
https://www.elastic.co/guide/en/beats/filebeat/current/faq.html#newline-character-required-eof

[Vivek_Sharma]

Thanks a lot for your help. My problem is solved now.

Actually I was not pressing enter key after last line of the log. So it was not picking up the last line.

Once I pressed enter key at the end of last line of log, the record was picked up.