Hi all,
I've defined some fields in logstash filter which is stored in /etc/logstash/conf.d/
The filter is as follows:
input {
beats {
port => 5044
}
}
filter {
if [type] == "log" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
match => { "message" => "%{LOGLEVEL:loglevel}\s+%{MONTHDAY}[./-]%{MONTHNUM}[./-]%{YEAR}%{SPACE}%{TIME:time}\,%{POSINT:just_no}\s+(\[%{WORD:module}\.py)\:%{POSINT:line_no}]\s+%{WORD:added-word} : %{WORD:method} : %{UUID:word} : %{WORD:service-type}: %{NUMBER:time}" }
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch {
hosts => ["xxx:9200","xxx:9200"] //private-ip
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
There is no error in either logstash error files nor rubydebug is showing any error.
I am able to get the message which is defined but I am not able to get the fields also defined.
Kibana just shows default fields.
Here is the screenshot for the Kibana dashboard
I've tried refreshing, other things. It would be great if anybody could help