Hi there! I'm totally new in ELK framework. I use Filebeat on CentOs 7 to ship logs to Logstash, here is my filebeat.yml:
filebeat.prospectors:
- type: log
paths:
- /opt/tomcat/logs/calendar/log.log
exclude_lines: ['^org.springframework.']
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 3
setup.kibana:
host: "localhost:5601"
output.logstash:
hosts: ["localhost:5044"]
Logstash .conf file:
input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{SYSLOG5424SD:thread} %{SYSLOGHOST:logger} \[%{JAVACLASS:class}\:%{NUMBER:javaline}\] %{GREEDYDATA:message}" }
overwrite => [ "message" ]
}
}
filter {
date {
match => [ "timestamp", ISO8601 ]
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "calendar7-%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
Yesterday everything was fine, I saw all logs in Kibana web-interface, logs were updated normally. But when I checked Kibana today, I saw nothing. Then I launched logstash with -f key and it's config file, new index was created and after that all log lines appeared in Kibana.
log.log file is rotated every 24hours: old log.log is renamed to log.[date].log, and new log.log file is created.
filebeat version 6.1.1
logstash version 6.1.1
Could you please help me to understand where is a problem?