Logstash config works but unable to see logs in Kibana

Hi. I'm trying to read two types of log files that one of them has different logs that one of them is in this type:
2019-06-26 01:00:31,069 INFO ir.ac.ut.sdrwebservice.SDRWebService @ batchAddStdDoc, System:G, User:25117, StudentIDs:[450188215], GroupID:4501, DocType:1349, returned 1561494631012101
and it is parsing correctly and I can see the related logs in discover part of Kibana.
but my problem is with the second file that just has logs in this type:
OQUEUE, Wed Jun 26 01:00:34 +0430 2019, 1561494631012101
I wrote a grok filter for it and it's tested with grok debugger but the related logs are not visible in discover part of Kibana.
here is my grok config filter:

  if([message] =~ /QUEUE,/){
    grok {
      match => {"message" => "%{NOTSPACE:queueType}, (?<nothing>.{4})(?<part1>.{15})(?<nothing2>.{6}) %{NUMBER:part2}, %{NOTSPACE:returnedCode}"}
    }
    mutate {
      add_field => {
        "timestamp1" => "%{part1} %{part2}"
      }
      remove_field => ["part1", "part2", "nothing", "nothing2"]
    }
    date {
      match => ["timestamp1", "MMM dd HH:mm:ss YYYY"]
    }
  }
  if ([message] !~ /batchAddStdDoc/) {
    drop { }
  }
  if ([message] !~ /returned/) {
    drop { }
  }
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:loglevel}  %{NOTSPACE:webService} @ %{NOTSPACE:function}, System:(?<systemName>.), User:%{NOTSPACE:userId}, StudentIDs:\[%{NUMBER:studentId}\], GroupID:%{GREEDYDATA:groupId}, DocType:%{NOTSPACE:docType}, returned %{INT:returnedCode}" }
  }
  date {
    match => ["timestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
  }
}

thanks for your help.

The message does not match either of the if conditions, so it will go through the

drop {}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.