I've run into an issue with newer versions of Winlogbeat that attempt to perform gap detection against the Windows channels.
First, create any XML query that results in gaps in the record IDs (this should hopefully work on most Windows systems):
PS C:\beats\winlogbeat-9.3.3-windows-x86_64> $xml = @'
>> <QueryList>
>> <Query Id="0">
>> <Select Path="Application">*[System[Provider[@Name='Windows Error Reporting'] and Level=4 and EventID=1001]]</Select>
>> </Query>
>> </QueryList>
>> '@
Then observe the gap (output cut down to the gap):
PS C:\beats\winlogbeat-9.3.3-windows-x86_64> Get-WinEvent -FilterXml $xml | Format-Table TimeCreated, RecordId, Level, Message
TimeCreated RecordId Level Message
----------- -------- ----- -------
4/3/2026 4:21:54 PM 546693 4 Fault bucket , type 0...
4/3/2026 4:21:54 PM 546692 4 Fault bucket , type 0...
Setup Winlogbeats to monitor the same query:
winlogbeat.event_logs:
- id: Application
xml_query: |-
<QueryList>
<Query Id="0">
<Select Path="Application">*[System[Provider[@Name='Windows Error Reporting'] and Level=4 and EventID=1001]]</Select>
</Query>
</QueryList>
output.file:
path: output
Run it and it will get "stuck" at the gap:
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.943-0400","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":833},"message":"Home`` path: [C:\beats\winlogbeat-9.3.3-windows-x86_64] Config path: [C:\beats\winlogbeat-9.3.3-windows-x86_64] Data path: [C:\beats\winlogbeat-9.3.3-windows-x86_64\data] Logs path: [C:\beats\winlogbeat-9.3.3-windows-x86_64\logs]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.946-0400","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":841},"message":"Beat`` ID: aef5ff15-649c-4c8d-bd9b-5505e4dbd681","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.952-0400","log.logger":"beat","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).createBeater","file.name":"instance/beat.go","file.line":333},"message":"Setup`` Beat: winlogbeat; Version: 9.3.3 (FIPS-distribution: false)","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.952-0400","log.logger":"beat","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1390},"message":"Beat`` info","service.name":"winlogbeat","system_info":{"beat":{"path":{"config":"C:\beats\winlogbeat-9.3.3-windows-x86_64","data":"C:\beats\winlogbeat-9.3.3-windows-x86_64\data","home":"C:\beats\winlogbeat-9.3.3-windows-x86_64","logs":"C:\beats\winlogbeat-9.3.3-windows-x86_64\logs"},"type":"winlogbeat","uuid":"aef5ff15-649c-4c8d-bd9b-5505e4dbd681"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.952-0400","log.logger":"beat","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1399},"message":"Build`` info","service.name":"winlogbeat","system_info":{"build":{"commit":"67e4444020f495415ad83b44a8508db7e5010fc1","libbeat":"9.3.3","time":"2026-04-01T19:12:08.000Z","version":"9.3.3"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.952-0400","log.logger":"beat","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1402},"message":"Go`` runtime info","service.name":"winlogbeat","system_info":{"go":{"os":"windows","arch":"amd64","max_procs":20,"version":"go1.25.8"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.955-0400","log.logger":"beat","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1408},"message":"Host`` info","service.name":"winlogbeat","system_info":{"host":{"architecture":"x86_64","native_architecture":"x86_64","boot_time":"2026-04-12T01:08:04-04:00","name":"DESKTOP-4KBAEKP","ip":[],"kernel_version":"10.0.26100.8117 (WinBuild.160101.0800)","mac":[],"os":{"type":"windows","family":"windows","platform":"windows","name":"Windows 11 Pro","version":"10.0","major":10,"minor":0,"patch":0,"build":"26200.8117"},"timezone":"XYZ","timezone_offset_sec":0,"id":""},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.955-0400","log.logger":"beat","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1437},"message":"Process`` info","service.name":"winlogbeat","system_info":{"process":{"cwd":"C:\beats\winlogbeat-9.3.3-windows-x86_64","exe":"C:\beats\winlogbeat-9.3.3-windows-x86_64\winlogbeat.exe","name":"winlogbeat.exe","pid":182280,"ppid":157560,"start_time":"2026-04-30T08:33:40.558-0400"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.962-0400","log.logger":"file","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/outputs/fileout.(*fileOutput).init","file.name":"fileout/file.go","file.line":107},"message":"Initialized`` file output. path=output\winlogbeat max_size_bytes=10485760 max_backups=7 permissions=-rw-------","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.962-0400","log.logger":"publisher","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/publisher/pipeline.LoadWithSettings","file.name":"pipeline/module.go","file.line":106},"message":"Beat`` name: DESKTOP-4KBAEKP","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.962-0400","log.logger":"winlogbeat","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/beater.New","file.name":"beater/winlogbeat.go","file.line":70},"message":"State`` will be read from and persisted to C:\beats\winlogbeat-9.3.3-windows-x86_64\data\.winlogbeat.yml","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.962-0400","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).launch","file.name":"instance/beat.go","file.line":539},"message":"winlogbeat`` start running.","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.962-0400","log.logger":"monitoring","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*Reporter).snapshotLoop","file.name":"log/log.go","file.line":145},"message":"Starting`` metrics logging every 30s","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2026-04-30T08:33:43.964-0400","log.logger":"winlogbeat","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/beater.(*Winlogbeat).Run","file.name":"beater/winlogbeat.go","file.line":154},"message":"Winlogbeat`` is unable to load the ingest pipelines because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines, you can ignore this warning.","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2026-04-30T08:33:43.964-0400","log.logger":"metric_registry","log.origin":{"function":"``github.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewDeprecatedMetricsRegistry","file.name":"inputmon/input.go","file.line":94},"message":"registering","service.name":"winlogbeat","input_type":"winlog","input_id":"Application","registry_id":"Application","ecs.version":"1.6.0``"}
{"log.level":"warn","@timestamp":"2026-04-30T08:33:43.978-0400","log.logger":"wineventlog","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/eventlog.(*winEventLog).processHandle","file.name":"eventlog/wineventlog.go","file.line":506},"message":"Record`` ID gap detected, resetting subscription.","service.name":"winlogbeat","id":"Application","channel":"","previous_record_id":546687,"current_record_id":546692,"missing":4,"ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2026-04-30T08:33:43.979-0400","log.logger":"winlogbeat","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/eventlog.Run.func4","file.name":"eventlog/runner.go","file.line":109},"message":"encountered`` recoverable error when reading from Windows Event Log","service.name":"winlogbeat","id":"Application","error":{"message":"record ID gap detected in channel "" (previous=546687 current=546692)"},"ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2026-04-30T08:33:48.990-0400","log.logger":"wineventlog","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/eventlog.(*winEventLog).processHandle","file.name":"eventlog/wineventlog.go","file.line":506},"message":"Record`` ID gap detected, resetting subscription.","service.name":"winlogbeat","id":"Application","channel":"","previous_record_id":546687,"current_record_id":546692,"missing":4,"ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2026-04-30T08:33:48.991-0400","log.logger":"winlogbeat","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/eventlog.Run.func4","file.name":"eventlog/runner.go","file.line":109},"message":"encountered`` recoverable error when reading from Windows Event Log","service.name":"winlogbeat","id":"Application","error":{"message":"record ID gap detected in channel "" (previous=546687 current=546692)"},"ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2026-04-30T08:33:59.001-0400","log.logger":"wineventlog","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/eventlog.(*winEventLog).processHandle","file.name":"eventlog/wineventlog.go","file.line":506},"message":"Record`` ID gap detected, resetting subscription.","service.name":"winlogbeat","id":"Application","channel":"","previous_record_id":546687,"current_record_id":546692,"missing":4,"ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2026-04-30T08:33:59.002-0400","log.logger":"winlogbeat","log.origin":{"function":"``github.com/elastic/beats/v7/winlogbeat/eventlog.Run.func4","file.name":"eventlog/runner.go","file.line":109},"message":"encountered`` recoverable error when reading from Windows Event Log","service.name":"winlogbeat","id":"Application","error":{"message":"record ID gap detected in channel "" (previous=546687 current=546692)"},"ecs.version":"1.6.0"}
This will likely happen forever. Looking at the stored bookmark is instructive:
update_time: 2026-04-30T13:16:52.592796Z
event_logs:
- name: Application
record_number: 546687
timestamp: 2026-04-03T20:08:36.2699756Z
bookmark: "<BookmarkList>\r\n <Bookmark Channel='Application' RecordId='546687'
IsCurrent='true'/>\r\n</BookmarkList>"
As best as I can tell, winlogbeat reads the record in the bookmark successfully, resets the gap detection, reads the record past the gap, and fails. Since reading the bookmarked record clears the retry count, it will never give up and just move on.
In any event, I believe the correct thing to do is to alter beats/winlogbeat/eventlog/wineventlog.go at 1a7f63b5e12174b52e4cc4fa85c1f82eaec1a427 · elastic/beats · GitHub to detect if an XML query is in use and just outright disable gap detection. Windows does not promise the record IDs will be contiguous when an XML query is involved. It doesn't even guarantee they will be in order according to their own docs.
I am aware for the query above that winlogbeat can do the filtering on its own. The actual query is more complicated and not something I can easily replicate with winlogbeat's filtering.
Additionally, fixing the retry for this case is not super desirable as it will harm performance. There are a lot of gaps and it would take a very long time for winlogbeat to process them all.