Logstash 2.4 output won't execute until stop


#1

Hi,

I am running logstash 2.4 on ubuntu 14.0.4.

My config file is:
input {
file {
path => "/home/spark/.log"
start_position => beginning
codec => multiline {
pattern => "(^\d+.
)"
what => "previous"
}
}
}
filter {
grok {
match => ["path","%{GREEDYDATA}/%{GREEDYDATA:filename}.log"]
}
}
output {
file {
path => "/home/spark/file"
}
azureblob {
storage_account_name => ""
storage_access_key => "
"
azure_container => "test"
}
}

I run logstash from command line:
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/first-pipeline.conf -l /var/log/logstash/logstash.log --verbose
In this way, I found the output will not actually execute until it receives SIGINT. What's the problem here?


(Mark Walkom) #2

Given you are using multiline it's probably waiting for a CRLF so that it know's the pattern has ended.


#3

I putted an enter character at the end of the file, it's still not work. BTW, I can see the position of the files are written in since_db file. It means logstash already got the input, but for some reason it doesn't handle it in the output. I guess it has bugs in multiline codec or logstash?


(Mark Walkom) #4

Just because something doesn't work the way you expect doesn't mean there are bugs.

Have a read of https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_tracking_of_current_position_in_watched_files


#5

I have read the file document, but I can't find the solution of my problem. I set the input start_position to beginning, the last modify time of the file not exceed ignore_older value and it has LF in the end. I believe multiline codec is able to pass the input to output, or it will be meaningless. How can I make it happen?


(Mark Walkom) #6

Did you try removing the sincedb?


#7

I found a solution of this issue. Enable auto flush by set a small value to auto_flush_interval in multiline codec. It will push events to output periodically.


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.