Winlogbeat configuration for index rollover

Elastic Stack Beats
1

Hello Everyone we have been testing a configuration for Winlogbeat for windows event log ingestion and with the out of the box configuration of about 60 machines we have noticed that we have over 70GB of data per month now we have 3 month retention and i create an ILM policy that deletes all indexes older than 3 months but what i have also noticed as behavior is that once the index is deleted Winlogbeat re-ingests old data (im assuming that is by design) and would just like some advise on how to approach the configuration to limit the old data being ingested

Version used is Winlogbeat-oss-07.12

configuration is below:


winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h

  - name: System

  - name: Security

  - name: ForwardedEvents
    tags: [forwarded]

  - name: Windows PowerShell
    event_id: 400, 403, 600, 800

  - name: Microsoft-Windows-PowerShell/Operational
    event_id: 4103, 4104, 4105, 4106

# ====================== Elasticsearch template settings =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

setup.template.name: "winlogbeat"
setup.template.pattern: "winlogbeat-*"

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["{{ elasticsearch_server }}:9200"]
  index: "winlogbeat-%{[agent.version]}-%{+yyyy.MM}"
  # Protocol - either `http` (default) or `https`.
  protocol: "https"

Any assistance would be most appreciated

2

No, this is not how it is designed. Winlogbeat use a local file to persist its state in the form of bookmarks. If you stop the Winlogbeat service you can inspect this file.

3

If you have new clients coming online and want to prevent them from sending older data then add the ignore_older option into each event log reader. You already have it there for the Application reader.