# Filebeat and Logstash with log rotation

**URL:** <https://discuss.elastic.co/t/filebeat-and-logstash-with-log-rotation/115866>\
**Category:** Logstash\
**Created:** [January 17, 2018, 11:01am UTC](https://discuss.elastic.co/t/filebeat-and-logstash-with-log-rotation/115866 "2018-01-17T11:01:15Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![uliana\_andreeva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/uliana_andreeva/32/26656_2.png) [@uliana\_andreeva](https://discuss.elastic.co/u/uliana_andreeva)\
**Post date:** [January 17, 2018, 11:01am UTC](https://discuss.elastic.co/t/filebeat-and-logstash-with-log-rotation/115866/1 "2018-01-17T11:01:15Z")

</div>

Hi there! I'm totally new in ELK framework. I use Filebeat on CentOs 7 to ship logs to Logstash, here is my filebeat.yml:

```
filebeat.prospectors:
    - type: log
      paths:
        - /opt/tomcat/logs/calendar/log.log
      exclude_lines: ['^org.springframework.']

    filebeat.config.modules:
      path: ${path.config}/modules.d/*.yml
      reload.enabled: false

    setup.template.settings:
      index.number_of_shards: 3

    setup.kibana:
      host: "localhost:5601"

    output.logstash:
      hosts: ["localhost:5044"]

```

Logstash .conf file:

```
input {
    beats {
        port => "5044"
    }
}
filter {
    grok {
      match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{SYSLOG5424SD:thread} %{SYSLOGHOST:logger} \[%{JAVACLASS:class}\:%{NUMBER:javaline}\] %{GREEDYDATA:message}" }
      overwrite => ["message"]
    }
}
filter {
    date {
      match => ["timestamp", ISO8601]
    }
}
output {
    elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "calendar7-%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    }
    stdout { codec => rubydebug }
}

```

Yesterday everything was fine, I saw all logs in Kibana web-interface, logs were updated normally. But when I checked Kibana today, I saw nothing. Then I launched logstash with -f key and it's config file, new index was created and after that all log lines appeared in Kibana.  
log.log file is rotated every 24hours: old log.log is renamed to log.[date].log, and new log.log file is created.

filebeat version 6.1.1  
logstash version 6.1.1

Could you please help me to understand where is a problem?

---

<div class="post-metadata">

**Author:** ![Kurt\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kurt_s/32/17755_2.png) [@Kurt\_S](https://discuss.elastic.co/u/Kurt_S)\
**Post date:** [January 17, 2018, 8:42pm UTC](https://discuss.elastic.co/t/filebeat-and-logstash-with-log-rotation/115866/2 "2018-01-17T20:42:00Z")

</div>

Was Logstash still running before you (re-)launched it?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 14, 2018, 8:42pm UTC](https://discuss.elastic.co/t/filebeat-and-logstash-with-log-rotation/115866/3 "2018-02-14T20:42:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.