Ingest only ERROR and warning logs to elastic search

[rahul_sirugudi]

Hi I am trying to ingest only ERROR and warning logs to Elasticsearch. Currently filebeat is shipping logs to log stash, i am using grok pattern.

grok {
match => { "message" => "%{IPV4:ip} - \[%{TIMESTAMP_ISO8601:timestamp}\] - %{GREEDYDATA:message} - %{GREEDYDATA:pool} - %{LOGLEVEL:log-level} : %{GREEDYDATA:error-message}" }
}
output {
if [fields][type] == "application_logs"
{
    elasticsearch {
    hosts => ["ip:9200"]
   user => "elastic"
   password => "password"
   index => "application-logs"
   }
 if "ERROR" in [log-level]
{
   stdout { codec => rubydebug }
  }

}
}

This above pattern is showing all logs including INFO. What can be done here?

[Badger]

Try wrapping your elasticsearch output in something like

if [log-level] in [ "ERROR", "WARN" ] {
    ...
}

[rahul_sirugudi]

Thanks Tried that, it worked i have to make few changes to file.

grok {
match => { "message" => "%{IPV4:ip} - \[%{TIMESTAMP_ISO8601:timestamp}\] - %{GREEDYDATA:message} - %{GREEDYDATA:pool} - %{LOGLEVEL:log-level} : %{GREEDYDATA:error-message}" }
}
output {
if [fields][type] == "application_logs"
{
 if [log-level] in [ "ERROR", "WARN" ]
{
   stdout { codec => rubydebug }
  }
    elasticsearch {
    hosts => ["ip:9200"]
   user => "elastic"
   password => "password"
   index => "application-logs"
   }


}
}
}