Logstash not parsing

Elastic Stack Logstash
1

After building a new machine of Logstash and copying the same exact "filter" used in the last machine with Logstash. From some reason Logstash is not parsing the logs correctly.

Info:
Filebeat is sending three types of logs 2 of them are not in json form to whom I expect Logstash to send them as raw material to ES. While one log type is by Json format which I expect Logstash to parse it.

The result of my LS in the ES is:

While the terms in message (underlined) should be by there self terms.

Here is my LogStash config file in /etc/logstash/conf.d (called 02-beats-input.conf)

input {
  beats {
    port => 5044
  }
}

filter {
  date {
    match => [ "msgSubmissionTime", "UNIX_MS" ]
    target => "msgSubmissionTime"
  }
  date {
    match => [ "msgDeliveryTime", "UNIX_MS" ]
    target => "msgDeliveryTime"
  }
  date {
    match => [ "eventTs", "UNIX_MS" ]
    target => "eventTs"
  }


  mutate {
    convert => {
        "concatenated" => "boolean"
        "msgLength" => "integer"
      }
   }

}


output {
  elasticsearch {
    hosts => ["192.168.1.116:9200"]
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

I installed the mutate and date plugin.

I also deleted the index in ES and restarted ES

Any help will be greatly helpful!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.