# Log rotation and filebeat

**URL:** <https://discuss.elastic.co/t/log-rotation-and-filebeat/140285>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 17, 2018, 8:47am UTC](https://discuss.elastic.co/t/log-rotation-and-filebeat/140285 "2018-07-17T08:47:31Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![bsikander](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@bsikander](https://discuss.elastic.co/u/bsikander)\
**Post date:** [July 17, 2018, 12:44pm UTC](https://discuss.elastic.co/t/log-rotation-and-filebeat/140285/3 "2018-07-17T12:44:31Z")

</div>

> > How fast are files created.  
> > it also depends on the loads. It is hard for me to give you exact numbers. In a problematic VM, we could have a new file almost every minute but with little data inside (~20MB).

> > How fast are files rotated (that is, how long until they reach 200M)  
> > Well, it varies. In some cases it could take around 3-4 hours.

> > Your current config.  
> > Here is our current config

```

    filebeat:
  prospectors:
    -
      document_type: jobA
      input_type: log
      paths:
        - "/path/to/jobA/logs/*/*.log"
      multiline:
         pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}'
         negate: true
         match: after
      fields:
          instance_name: ""
          instance_id: ""

    -
      document_type: jobB
      input_type: log
      paths:
        - "/path/to/jobB/logs/*/*/std*"
        - "/path/to/jobB/logs2/*/std*"
      multiline:
         pattern: '(^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3})|(^[0-9]{2}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2})'
         negate: true
         match: after
      fields:
          instance_name: ""
          instance_id: ""

  registry_file: /path/to/filebeat/registry

output:
  logstash:
    hosts: [":5044"]
    bulk_max_size: 1024

shipper:

logging:
  to_syslog: false
  to_files: true
  files:
    path: /path/to/filebeat/log
    name: filebeat.log
    rotateeverybytes: 10485760 # = 10MB
    keepfiles: 7

```

---

_[View the full topic](https://discuss.elastic.co/t/log-rotation-and-filebeat/140285)._