Logstash file input plugin no output response

I am new to ELK so any comment is welcome.

I am now using logstash to parse logs and I want to combine different lines of into one line by using codec => multiline plugin. Here is my config.

input {
  #beats {
  #  port => 5044
  #}
  file {
    path => "C:\ELK\log_190712\mq.log"
    start_position => "beginning"
    codec => multiline {
      # grok pattern are valid! :)
      pattern => "^(?<logTimestamp>\d{2} \w{3} \d{4} \d{2}:\d{2}:\d{2},\d{3})"
      negate => true
      what => "previous"
    }
  }
}
filter {
  grok {		
     match => [ "message", "(?<logTimestamp>\d{2} \w{3} \d{4} \d{2}:\d{2}:\d{2},\d{3}) %{DATA:logType} %{DATA:ip} \[%{NUMBER:random}\] \[%{DATA:username}\] \[\[%{DATA:threadstate}\] %{DATA:thread}\] %{DATA:logger} %{GREEDYDATA:detail}" ]
  }
  mutate {
    add_tag => ["logstash"]
  }
  date {
    match => [ "logTimestamp", "dd MMM yyyy HH:mm:ss,SSS" ]
    target => "logTimestamp"
  }  
}
output {
  
  # file {
  #   path => "C:\ELK\mq_change.log"
	#   codec => line { format => "custom format: %{message}"}
  #   #message_format => "%{message}"
  # }
  stdout {
      codec => rubydebug
  }
}

I have tested the codec => multiline plugin using stdin and it works fine, but when I switch to file plugin, then no responsive even using debug mode. I have tried to delete .sincedb and i also don't work.

The debug message looks like this,

Please take a look at my case and help me with this problem. Thx.

Firstly, use forward slash instead of backslash in a file input.

Secondly, when you use 'negate => true' an event is not pushed until there is a second line that does match the pattern. So you may need to use a timeout, such as 'auto_flush_interval => 5'.

Thanks for your comment :).

I have changed that but it didn't work. Finally, by fixing the path and adding timeout mechanism plus adding 'mode = "read"', it finally run as I hope. But I don't know why it works and how it works. Can you explain about it. Thanks.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.