Logstash 7.4.0 "grok " filter not work or might some problem with Logstash

Here, I have tried to send MySQL's error log data through filebeat 7.4.0 to Logstash 7.4.0 and after filter, the required information, send these all data to Elastic and then viewing through Kibana.

There has some problems that error log data did not filter with Logstash's config file and the whole line of error log processed and displayed to Kibana without any error in Logstash and elastic.

Software versions:

MySQL 8.0.18 GPL
ELK - 7.4.0
Filebeat - 7.4.0

The requirement to use Logstash: I need the only word "NOTE|WARNING|ERRRO" from error log statement when any problem with MySQL

Here I displayed Logstash and Filebeat

##################Logstash config file#########
input {
  beats {
    port => 5044
    host => "XXXXXXX"
  }
}


filter {
    if [fileset][module] == "mysql" {
    if [fileset][name] == "error" {

         grok {


                  match => ["message", "(?<errortype>(Note|Warning|ERROR))"]

                  add_field => { "errortype" => "%{errortype}" }
        }
}
}

}



output{

elasticsearch {
    hosts => "XXXXXXX:9200"
    manage_template => false
    index => "filebeatindex"
  }
}
########################################

######################Filebeat config file#############
filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
          
    - /file-path-of-mysql-instance-1/err1.log
    - /file-path-of-mysql-instance-2/err2.log
    - /file-path-of-mysql-instance-3/err3.log
    #- c:\programdata\elasticsearch\logs\*




#================================ Outputs =====================================
output.logstash:
  # The Logstash hosts
  hosts: ["XXXXXXX:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

Can anyone suggest to me what I have to do to solve this?
Am I used correct version?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.