Logstash config works but unable to see logs in Kibana

Elastic Stack Logstash
1

Hi. I'm trying to read two types of log files that one of them has different logs that one of them is in this type:
2019-06-26 01:00:31,069 INFO ir.ac.ut.sdrwebservice.SDRWebService @ batchAddStdDoc, System:G, User:25117, StudentIDs:[450188215], GroupID:4501, DocType:1349, returned 1561494631012101
and it is parsing correctly and I can see the related logs in discover part of Kibana.
but my problem is with the second file that just has logs in this type:
OQUEUE, Wed Jun 26 01:00:34 +0430 2019, 1561494631012101
I wrote a grok filter for it and it's tested with grok debugger but the related logs are not visible in discover part of Kibana.
here is my grok config filter:

  if([message] =~ /QUEUE,/){
    grok {
      match => {"message" => "%{NOTSPACE:queueType}, (?<nothing>.{4})(?<part1>.{15})(?<nothing2>.{6}) %{NUMBER:part2}, %{NOTSPACE:returnedCode}"}
    }
    mutate {
      add_field => {
        "timestamp1" => "%{part1} %{part2}"
      }
      remove_field => ["part1", "part2", "nothing", "nothing2"]
    }
    date {
      match => ["timestamp1", "MMM dd HH:mm:ss YYYY"]
    }
  }
  if ([message] !~ /batchAddStdDoc/) {
    drop { }
  }
  if ([message] !~ /returned/) {
    drop { }
  }
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:loglevel}  %{NOTSPACE:webService} @ %{NOTSPACE:function}, System:(?<systemName>.), User:%{NOTSPACE:userId}, StudentIDs:\[%{NUMBER:studentId}\], GroupID:%{GREEDYDATA:groupId}, DocType:%{NOTSPACE:docType}, returned %{INT:returnedCode}" }
  }
  date {
    match => ["timestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
  }
}

thanks for your help.

2

The message does not match either of the if conditions, so it will go through the

drop {}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.