# Audit logs are not rotated correctly when the elasticsearch module is enabled **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [August 2, 2019, 12:01pm UTC](https://discuss.elastic.co/t/audit-logs-are-not-rotated-correctly-when-the-elasticsearch-module-is-enabled/193526 "2019-08-02T12:01:42Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![gerarddp](https://avatars.discourse-cdn.com/v4/letter/g/839c29/32.png) [@gerarddp](https://discuss.elastic.co/u/gerarddp)\ **Post date:** [August 2, 2019, 12:01pm UTC](https://discuss.elastic.co/t/audit-logs-are-not-rotated-correctly-when-the-elasticsearch-module-is-enabled/193526/1 "2019-08-02T12:01:42Z")
When enabling the audit log parsing with the elasticsearch module for filbeat, the audit logs are not rotate correctly because filebeat keeps the files open. That also causes the disk to get full quickly because of the amount of audit logs. Is there a way to keep using the module without this issues? ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 5, 2019, 10:59am UTC](https://discuss.elastic.co/t/audit-logs-are-not-rotated-correctly-when-the-elasticsearch-module-is-enabled/193526/2 "2019-08-05T10:59:33Z")
The issue is not coming from using the module. The root cause is that the output is processing the events from Filebeat too slowly. Do you have any errors in the logs of the output? Also, you can set the appropiate `close_*` option, so Filebeat closes files: [https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#close-options](https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#close-options) As you are using a module, you need to set it under `input`: ```auto #---------------------------- Elasticsearch Module ---------------------------- - module: elasticsearch server: enabled: true input: close_removed: true ``` Note that I am not suggesting that `close_removed` is something you need to enable. The options to use/enable depend on your logs and logrotation. ---
**Author:** ![gerarddp](https://avatars.discourse-cdn.com/v4/letter/g/839c29/32.png) [@gerarddp](https://discuss.elastic.co/u/gerarddp)\ **Post date:** [August 5, 2019, 2:14pm UTC](https://discuss.elastic.co/t/audit-logs-are-not-rotated-correctly-when-the-elasticsearch-module-is-enabled/193526/3 "2019-08-05T14:14:18Z")
I've enabled `close_removed` but the issue is not fixed. Also the issue is with the audit part, not server. All the others work correctly, but there's not as many logs as the audit. I've tried searching for any errors, but I cannot see any. The output is elasticsearch itself, so maybe there's a loop issue? Maybe this will help (with `close_removed: true`) ``` #lsof -p 32060 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME [...] filebeat 32060 root 6r REG 254,1 268435773 262779 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 7r REG 254,1 268435627 262783 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 8r REG 254,1 268435670 262784 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 9r REG 254,1 268435549 262785 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 10r REG 254,1 268435673 263014 /var/log/elasticsearch/logging_audit-2019-08-05-3.json filebeat 32060 root 11r REG 254,1 268435528 262786 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 12r REG 254,1 268435505 262926 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 13r REG 254,1 40754471 262789 /var/log/elasticsearch/gc.log filebeat 32060 root 14r REG 254,1 40754471 262789 /var/log/elasticsearch/gc.log filebeat 32060 root 15r REG 254,1 268435663 262928 /var/log/elasticsearch/igp-logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 16r REG 254,1 268435496 262976 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 17r REG 254,1 268435484 263115 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 18r REG 254,1 268435845 263565 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 19r REG 254,1 268435862 263568 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 20r REG 254,1 268435533 263573 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 21r REG 254,1 268435853 263574 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 22r REG 254,1 268435705 263575 /var/log/elasticsearch/logging_audit-2019-08-05-1.json (deleted) filebeat 32060 root 23r REG 254,1 268435801 263576 /var/log/elasticsearch/logging_audit-2019-08-05-1.json filebeat 32060 root 24r REG 254,1 268435853 263577 /var/log/elasticsearch/logging_audit-2019-08-05-2.json filebeat 32060 root 25r REG 254,1 268435847 263578 /var/log/elasticsearch/logging_audit-2019-08-05-4.json filebeat 32060 root 26r REG 254,1 268435539 263579 /var/log/elasticsearch/logging_audit-2019-08-05-5.json filebeat 32060 root 27r REG 254,1 68181742 262782 /var/log/elasticsearch/logging_audit.json ``` ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 6, 2019, 12:18pm UTC](https://discuss.elastic.co/t/audit-logs-are-not-rotated-correctly-when-the-elasticsearch-module-is-enabled/193526/4 "2019-08-06T12:18:00Z")
What do you use to rotate your logs? Could you share its configuration? ---
**Author:** ![gerarddp](https://avatars.discourse-cdn.com/v4/letter/g/839c29/32.png) [@gerarddp](https://discuss.elastic.co/u/gerarddp)\ **Post date:** [August 7, 2019, 1:56pm UTC](https://discuss.elastic.co/t/audit-logs-are-not-rotated-correctly-when-the-elasticsearch-module-is-enabled/193526/5 "2019-08-07T13:56:46Z")
We didn't change any configuration, the rotation is done by elasticsearch. We just enabled audit logs. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 4, 2019, 1:57pm UTC](https://discuss.elastic.co/t/audit-logs-are-not-rotated-correctly-when-the-elasticsearch-module-is-enabled/193526/6 "2019-09-04T13:57:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.