Winlogbeat sending data to default index

vee · January 12, 2022, 9:51pm

Trying to setup winlogbeat but it seems to be sending documents only to default index - winlogbeat-7* instead of the one I explicity mentioned in the winlogbeat.yml. I must be missing something basic but can't figure out...hope one of you guys can catch what's the issue here:

winglogbeat v: 7.13.2

winlogbeat.yml:

winlogbeat.event_logs:
  - name: FedEx    
    level: critical, error, warning
  - name: System
    level: critical, error, warning

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["es1.company.com:9200","es2.company.com:9200"]         

  # Optional protocol and basic auth credentials.
  protocol: "http"
  password: ***
  username: elastic 
  index: "uat-winlogbeat"
setup.template.enabled: false  
setup.template.name: "uat-winlogbeat"
setup.template.pattern: "uat-winlogbeat-*"

Here's the template definition in elastic:

{
  "uat-winlogbeat" : {
    "order" : 200,
    "index_patterns" : [
      "uat-winlogbeat-*"
    ],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "Small_ILM_Policy",
          "rollover_alias" : "uat-winlogbeat"
        },
        "refresh_interval" : "10s",
        "number_of_shards" : "1",
        "number_of_replicas" : "1"
      }
    },
    "mappings" : { },
    "aliases" : { }
  }
}

Alias:

{
  "uat-winlogbeat-2022.01.12-1" : {
    "aliases" : {
      "uat-winlogbeat" : {
        "is_write_index" : true
      }
    }
  }
}

Winlogbeat logs:

PS C:\Program Files\winlogbeat-7.13.2-windows-x86_64> .\winlogbeat.exe -e -c .\winlogbeat.yml
2022-01-12T15:39:15.508-0600    INFO    instance/beat.go:665    Home path: [C:\Program Files\winlogbeat-7.13.2-windows-x
86_64] Config path: [C:\Program Files\winlogbeat-7.13.2-windows-x86_64] Data path: [C:\Program Files\winlogbeat-7.13.2-w
indows-x86_64\data] Logs path: [C:\Program Files\winlogbeat-7.13.2-windows-x86_64\logs]
2022-01-12T15:39:15.520-0600    INFO    instance/beat.go:673    Beat ID: 30dc3d35-1a28-4f3b-afe9-3b2e08c82c88
2022-01-12T15:39:15.522-0600    INFO    [beat]  instance/beat.go:1014   Beat info       {"system_info": {"beat": {"path"
: {"config": "C:\\Program Files\\winlogbeat-7.13.2-windows-x86_64", "data": "C:\\Program Files\\winlogbeat-7.13.2-window
s-x86_64\\data", "home": "C:\\Program Files\\winlogbeat-7.13.2-windows-x86_64", "logs": "C:\\Program Files\\winlogbeat-7
.13.2-windows-x86_64\\logs"}, "type": "winlogbeat", "uuid": "30dc3d35-1a28-4f3b-afe9-3b2e08c82c88"}}}
2022-01-12T15:39:15.523-0600    INFO    [beat]  instance/beat.go:1023   Build info      {"system_info": {"build": {"comm
it": "686ba416a74193f2e69dcfa2eb142f4364a79307", "libbeat": "7.13.2", "time": "2021-06-10T21:23:44.000Z", "version": "7.
13.2"}}}
2022-01-12T15:39:15.524-0600    INFO    [beat]  instance/beat.go:1026   Go runtime info {"system_info": {"go": {"os":"wi
ndows","arch":"amd64","max_procs":8,"version":"go1.15.13"}}}
2022-01-12T15:39:15.528-0600    INFO    [beat]  instance/beat.go:1030   Host info       {"system_info": {"host": {"archi
tecture":"x86_64","boot_time":"2022-01-09T03:00:31.72-06:00","name":"host1","ip":["fe80::44c2:867:ab90:c596/64","
10.35.80.207/20","::1/128","127.0.0.1/8"],"kernel_version":"6.3.9600.20111 (winblue_ltsb_escrow.210812-0920)","mac":["00
:50:56:a3:df:58","00:00:00:00:00:00:00:e0"],"os":{"type":"windows","family":"windows","platform":"windows","name":"Windo
ws Server 2012 R2 Standard","version":"6.3","major":3,"minor":0,"patch":0,"build":"9600.20120"},"timezone":"CST","timezo
ne_offset_sec":-21600,"id":"b4046fd3-5756-4650-85c6-052ebfd71223"}}}
2022-01-12T15:39:15.530-0600    INFO    [beat]  instance/beat.go:1059   Process info    {"system_info": {"process": {"cw
d": "C:\\Program Files\\winlogbeat-7.13.2-windows-x86_64", "exe": "C:\\Program Files\\winlogbeat-7.13.2-windows-x86_64\\
winlogbeat.exe", "name": "winlogbeat.exe", "pid": 5456, "ppid": 4652, "start_time": "2022-01-12T15:39:14.974-0600"}}}
2022-01-12T15:39:15.530-0600    INFO    instance/beat.go:309    Setup Beat: winlogbeat; Version: 7.13.2
2022-01-12T15:39:15.531-0600    INFO    [index-management]      idxmgmt/std.go:184      Set output.elasticsearch.index t
o 'winlogbeat-7.13.2' as ILM is enabled.
2022-01-12T15:39:15.531-0600    INFO    eslegclient/connection.go:99    elasticsearch url: http://es1:9200
2022-01-12T15:39:15.531-0600    INFO    eslegclient/connection.go:99    elasticsearch url: http://es2:9200
2022-01-12T15:39:15.532-0600    INFO    [publisher]     pipeline/module.go:113  Beat name: host1
2022-01-12T15:39:15.532-0600    INFO    beater/winlogbeat.go:69 State will be read from and persisted to C:\Program File
s\winlogbeat-7.13.2-windows-x86_64\data\.winlogbeat.yml
2022-01-12T15:39:15.533-0600    INFO    [monitoring]    log/log.go:117  Starting metrics logging every 30s
2022-01-12T15:39:15.533-0600    INFO    instance/beat.go:473    winlogbeat start running.
2022-01-12T15:39:15.543-0600    WARN    beater/eventlogger.go:124       EventLog[FedEx] Open() error. No events will be
read from this source. The specified channel could not be found. Check channel configuration.
2022-01-12T15:39:16.618-0600    INFO    [publisher_pipeline_output]     pipeline/output.go:143  Connecting to backoff(el
asticsearch(http://es1com:9200))
2022-01-12T15:39:16.618-0600    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2022-01-12T15:39:16.618-0600    INFO    [publisher_pipeline_output]     pipeline/output.go:143  Connecting to backoff(el
asticsearch(http://es2.com:9200))
2022-01-12T15:39:16.620-0600    INFO    [publisher]     pipeline/retry.go:223     done
2022-01-12T15:39:16.621-0600    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2022-01-12T15:39:16.621-0600    INFO    [publisher]     pipeline/retry.go:223     done
2022-01-12T15:39:16.627-0600    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elastic
search version 7.13.2
2022-01-12T15:39:16.627-0600    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elastic
search version 7.13.2
2022-01-12T15:39:16.633-0600    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elastic
search version 7.13.2
2022-01-12T15:39:16.635-0600    INFO    [index-management]      idxmgmt/std.go:261      Auto ILM enable success.
2022-01-12T15:39:16.637-0600    INFO    [index-management.ilm]  ilm/std.go:160  ILM policy winlogbeat exists already.
2022-01-12T15:39:18.873-0600    INFO    [index-management.ilm]  ilm/std.go:135  Index Alias winlogbeat-7.13.2 successful
ly created.
2022-01-12T15:39:18.874-0600    INFO    [publisher_pipeline_output]     pipeline/output.go:151  Connection to backoff(el
asticsearch(http://es1com:9200)) established
2022-01-12T15:39:18.878-0600    INFO    [index-management]      idxmgmt/std.go:261      Auto ILM enable success.
2022-01-12T15:39:18.881-0600    INFO    [index-management.ilm]  ilm/std.go:160  ILM policy winlogbeat exists already.
2022-01-12T15:39:18.889-0600    INFO    [index-management.ilm]  ilm/std.go:121  Index Alias winlogbeat-7.13.2 exists alr
eady.
2022-01-12T15:39:18.890-0600    INFO    [publisher_pipeline_output]     pipeline/output.go:151  Connection to backoff(el
asticsearch(http://es2.com:9200)) established
2022-01-12T15:39:19.164-0600    INFO    beater/eventlogger.go:88        EventLog[System] successfully published 50 event
s
2022-01-12T15:39:19.437-0600    INFO    beater/eventlogger.go:88        EventLog[System] successfully published 100 even
ts
2022-01-12T15:39:19.497-0600    INFO    beater/eventlogger.go:88        EventLog[System] successfully published 50 event
s
2022-01-12T15:39:19.576-0600    INFO    beater/eventlogger.go:88        EventLog[System] successfully published 50 event
s
2022-01-12T15:39:19.599-0600    INFO    beater/eventlogger.go:88        EventLog[System] successfully published 50 event
s
2022-01-12T15:39:19.624-0600    INFO    beater/eventlogger.go:88        EventLog[System] successfully published 50 event
s

I see data going into winlogbeat-7.13.2-* index as opposed to the one listed in the above yml file.
Thanks!

andrewkroh · January 13, 2022, 2:53am

It looks like you are managing the ILM setup and write alias creation yourself so disable ILM setup in the Beats.

See Configure index lifecycle management | Filebeat Reference [7.16] | Elastic

setup.ilm.enabled: false

vee · January 13, 2022, 4:00pm

Thanks Andrew! That seems to have worked, although am confused as to why I have to specify that property on winlogbeat even though I do plan on using ILM to manage. I thought alias creation with the seed index as write index is needed to start using ILM.

system · February 10, 2022, 6:00pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Shipping Logs to Logstash not Working Beats winlogbeat	4	1460	November 3, 2017
Unable to change the default index in Winlogbeat.yaml Beats winlogbeat	15	2204	January 2, 2018
Winlogbeat dont use custom template Beats winlogbeat	1	556	December 5, 2019
Winlogbeat sends not all eventlogs Beats winlogbeat	2	347	March 24, 2020
Winlogbeat still creates default index when setting up to use a custom index Beats winlogbeat	1	490	July 20, 2022

Winlogbeat sending data to default index

Related topics