# Elasticsearch & Kibana Audit logs **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [November 17, 2022, 8:46am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151 "2022-11-17T08:46:53Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 17, 2022, 8:46am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/1 "2022-11-17T08:46:53Z")
Hi Team, Elasticsearch audit logs is taking 20 gb size everyday. Can you please help me to minimize the size of audit log file. Regards, Syed ---
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 17, 2022, 8:52am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/2 "2022-11-17T08:52:33Z")
Hi Team, We are using elk & Kibana 7.16.2 version and same version for filebeat also. we have enabled the audit logs by enabling audit keys in elasticsearch.yml, But it is taking 20gb size per day. Requesting you to please help in minimizing the size of audit logs files. Regards, Syed ---
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 20, 2022, 2:15pm UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/3 "2022-11-20T14:15:37Z")
Hi Elastic Support Team, Can we get any updates on the same. Regards, Syed ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [November 20, 2022, 5:11pm UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/4 "2022-11-20T17:11:50Z")
Hi @syed0510 Yes auditing can be quite extensive please refer to these 2 document pages Configuring Auditing > **[Auditing security settings | Elasticsearch Guide \[7.16\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/auditing-settings.html)** Event Types > **[Audit events | Elasticsearch Guide \[7.16\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/audit-event-types.html)** And you can decide which events types you want to include and / or exclude in your audit logs. using `xpack.security.audit.logfile.events.include` and / or `xpack.security.audit.logfile.events.exclude` Or you can completely disable it if you do not need it. ---
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 20, 2022, 8:32pm UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/5 "2022-11-20T20:32:09Z")
Hi @stephenb , Thanks for the reply! I will check the document that you have shared and will update you accordingly. Regards, Syed ---
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 20, 2022, 8:51pm UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/6 "2022-11-20T20:51:45Z")
Hi @stephenb, Can you please resolve below queries- 1. Is there any way to set the log entry format of audit events attribute? 2. How can we use **xpack.security.audit.logfile.events.include** if we wants to get the list of all users who logged successfully. Thanks! Regards, Syed ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [November 20, 2022, 10:25pm UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/7 "2022-11-20T22:25:58Z")
> [@syed0510](#): > > Is there any way to set the log entry format of audit events attribute? There may be but it is a highly structure json log following ECS. It is very easy to read and there is even a module to load it into elasticsearc (see below). You would need to read / adjust the logging output in the `config/log4j2.properties` file which is not my area of expertise. If you change it then > [@syed0510](#): > > How can we use **xpack.security.audit.logfile.events.include** if we wants to get the list of all users who logged successfully. quick look at the docs look like it would look like `xpack.security.audit.logfile.events.include: ["authentication_success"]` That would log each successful login. Then you would read those logs into elasticsearch and visualize the results using... Perhaps take a look at > **[Indexing Elasticsearch Audit Logs with Filebeat](https://www.elastic.co/blog/indexing-elasticsearch-audit-logs-with-filebeat)** > > An effective way to analyze Elasticsearch audit logs from a security perspective is to index them into an Elasticsearch cluster. Starting with 7.0, the preferred method of indexing audit logs is to use Filebeat. This blog post explains the rationale... ---
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 21, 2022, 4:08am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/8 "2022-11-21T04:08:44Z")
Hi @stephenb, Thanks for the update! > [@stephenb](#): > > [Indexing Elasticsearch Audit Logs with Filebeat | Elastic Blog](https://www.elastic.co/blog/indexing-elasticsearch-audit-logs-with-filebeat) We are already using filebeat that's why audit logs size reached to 20 gb. Please find below the filebeat input file setting from filebeat.yml- ```auto # ============================== Filebeat inputs =============================== filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. # filestream is an input for collecting log messages from files. - type: filestream # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: #- /var/log/*.log - D:\Elastic\logs\elasticsearch_audit-*.json ``` #------------------------------------------- Requesting you to please help in minimize the size of audit log file through filebeat. Regards, Syed ---
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 21, 2022, 4:10am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/9 "2022-11-21T04:10:30Z")
Hi @stephenb , we have installed filebeat on the same server where elk installed and given the path of elk audit log file in filebeat.yml directly. Regards, Syed ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [November 21, 2022, 4:13am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/10 "2022-11-21T04:13:29Z")
I given you direction To only log login event's. Filebeat has nothing to do with the size of the logs. It is unclear what you want. Exactly what is 20GB the audit logfile? The index of the audit logs in elasticsearch? Everything you need is in the docs that I have provided. > **[Logging | Elasticsearch Guide \[8.5\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/logging.html)** Settings for how long to retain.. plus I showed you how to only log the login event's... Best I can do... Read the docs adjust.. test... adjust. ---
**Author:** ![syed0510](https://avatars.discourse-cdn.com/v4/letter/s/76d3ee/32.png) [@syed0510](https://discuss.elastic.co/u/syed0510)\ **Post date:** [November 21, 2022, 4:35am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/11 "2022-11-21T04:35:17Z")
Hi @stephenb, Thanks for the support! Actually I only wants to know here about the path of log that provided in filebet.yml is correct or not as we have given the path of elk audit log file in filebeat.yml directly. > Blockquote > [@syed0510](#): > > ```auto > # ============================== Filebeat inputs =============================== > > filebeat.inputs: > > # Each - is an input. Most options can be set at the input level, so > # you can use different inputs for various configurations. > # Below are the input specific configurations. > > # filestream is an input for collecting log messages from files. > - type: filestream > > # Change to true to enable this input configuration. > enabled: true > > # Paths that should be crawled and fetched. Glob based paths. > paths: > #- /var/log/*.log > - D:\Elastic\logs\elasticsearch_audit-*.json > > ``` Regards, Syed ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [November 21, 2022, 5:47am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/12 "2022-11-21T05:47:37Z")
> **[Install Elasticsearch with .zip on Windows | Elasticsearch Guide \[8.5\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/zip-windows.html#windows-layout)** Depends on where you installed elasticsearch, did you look are the logs there? There is a filebeat module that knows how to parse the audit logs. > **[Elasticsearch module | Filebeat Reference \[8.5\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-elasticsearch.html#_audit_log_fileset_settings_2)** ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 19, 2022, 5:48am UTC](https://discuss.elastic.co/t/elasticsearch-kibana-audit-logs/319151/13 "2022-12-19T05:48:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.