Filebeat and Logstash with log rotation

Hi there! I'm totally new in ELK framework. I use Filebeat on CentOs 7 to ship logs to Logstash, here is my filebeat.yml:

    - type: log
        - /opt/tomcat/logs/calendar/log.log
      exclude_lines: ['^org.springframework.']

      path: ${path.config}/modules.d/*.yml
      reload.enabled: false

      index.number_of_shards: 3

      host: "localhost:5601"

      hosts: ["localhost:5044"]

Logstash .conf file:

input {
    beats {
        port => "5044"
filter {
    grok {
      match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{SYSLOG5424SD:thread} %{SYSLOGHOST:logger} \[%{JAVACLASS:class}\:%{NUMBER:javaline}\] %{GREEDYDATA:message}" }
      overwrite => [ "message" ]
filter {
    date {
      match => [ "timestamp", ISO8601 ]
output {
    elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "calendar7-%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    stdout { codec => rubydebug }

Yesterday everything was fine, I saw all logs in Kibana web-interface, logs were updated normally. But when I checked Kibana today, I saw nothing. Then I launched logstash with -f key and it's config file, new index was created and after that all log lines appeared in Kibana.
log.log file is rotated every 24hours: old log.log is renamed to log.[date].log, and new log.log file is created.

filebeat version 6.1.1
logstash version 6.1.1

Could you please help me to understand where is a problem?

Was Logstash still running before you (re-)launched it?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.