Grok filter not working

Elastic Stack Logstash
1

LogFile

[0808 16:08:51 EAT] INFO 47260019218176 profile | Start Statistics Info
[vitual server] = 10.6.2.38:8080
recv called count = 1620

input {
file {
path => "/home/babukuse/data/profile.log.15"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "[%{SYSLOGTIMESTAMP:syslog_timestamp}][%{WORD:virtualserver}] %{IP:virtualserverIP} %{WORD:recv called count}%{NUMBER:request}" }
}
}
output {
elasticsearch { hosts => ["10.184.8.113:9200"]
index =>["machine4"]}
stdout { }
}

Output.
{
"host" => "svdt5cbs55ml.safaricom.net",
"@version" => "1",
"path" => "/home/babukuse/data/profile.log.15",
"@timestamp" => 2018-08-16T23:37:06.726Z,
"tags" => [
[0] "_grokparsefailure"
],
"message" => "[vitual server] = 10.6.2.38:8080"
}
{
"host" => "svdt5cbs55ml.safaricom.net",
"@version" => "1",
"path" => "/home/babukuse/data/profile.log.15",
"@timestamp" => 2018-08-16T23:37:06.704Z,
"tags" => [
[0] "_grokparsefailure"
],
"message" => "[0808 16:08:51 EAT] INFO 47260019218176 profile | Start Statistics Info"
}
{
"host" => "svdt5cbs55ml.safaricom.net",
"@version" => "1",
"path" => "/home/babukuse/data/profile.log.15",
"@timestamp" => 2018-08-16T23:37:06.727Z,
"tags" => [
[0] "_grokparsefailure"
],
"message" => "recv called count = 1620 "
}

2

By default the file input reads lines one by one. If you want to join multiple physical lines into a single event you need to use a multiline codec.

3

Hi,

I used almost the same filter in below file path and it worked, what makes the two files different?

55.3.244.1 GET /index.html 15824 0.043

input {
file {
path => "/home/babukuse/data/profile.log.15"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" }
}
}
output {
elasticsearch { hosts => ["10.184.8.113:9200"]
index =>["machine4"]}
stdout { }

Please advise

4

I don't understand what you mean. Those are two completely different log files with completely different grok filters. One of the filters matches its input log file and the other one doesn't.