Using Winlogbeat 7.2.0 with ILM enabled - Winlogbeat does not synchronize old logs. just todays logs

Hello,
I can't import the older logs into elasticsearch even though the following options have been specified into winlogbeat.yml.

winlogbeat.event_logs:
  - name: Application
    ignore_older: 720h
  - name: System
    ignore_older: 720h
  - name: Security
    ignore_older: 720h
    tags: ["security", "elastic1"]
winlogbeat.shutdown_timeout: 30s

The ILM settings are shown below:

# ILM and template settings
setup.template.overwrite: true
setup.ilm.enabled: true
setup.ilm.rollover_alias: "elastic1"
setup.ilm.pattern: "{now/d}-000001"
setup.ilm.policy_file: "ilm_policy.json"
setup.ilm.policy_name: "elastic1"
setup.ilm.overwrite: true

ilm_policy.json

{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "7d",
            "max_size": "10G"
          }
        }
      },
      "warm": {
        "min_age": "7d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 6
          },
          "shrink": {
            "number_of_shards": 6
          },
          "allocate": {
            "number_of_replicas": 1
          }
        }
      },
      "cold": {
        "min_age": "30d",
        "actions": {
          "allocate": {
            "require": {
              "type": "cold"
            }
          }
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Prior to ILM, logs from a past date, say 2019.01.01, would be placed into an index with the same name. With ILM, they go into the current writing index. But, the @timestamp should have the 2019.01.01 date.

Are you looking at the index name or the @timestamp?

1 Like

I just happened to install and start winlogbeat on a system sending to a test stack yesterday. I just checked and I have logs back to May, about the time the windows system was installed.

My winlogbeat version is 7.3.0, so close.

This is my mistake. The password in the Keystore had not been updated even though the password had been updated in kibana - Which for some reason was causing an indeterminate number of log entries to go through, which is strange.