Winlogbeat freezes

Elastic Stack Beats
1

Hello everyone,

I recently came across the topic of Elastic Search, following the motto: "Can you take a look at this" We have been having an issue since November where the transmission on our Active Directory Domain Controllers freezes after some time. Unfortunately, I am not getting any useful information from the log files that could help me solve the problem. Could you give me a few tips here? Windows Server 2022 Elastic Kibana v 8.14.3 Winlogbeat v8.16.1

2

Can you share your winlogbeat configuration as well as the log?

3 
# ======================== Winlogbeat specific options =========================
winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h

  - name: System

  - name: Security

  - name: Microsoft-Windows-Sysmon/Operational

  - name: Windows PowerShell
    event_id: 400, 403, 600, 800

  - name: Microsoft-Windows-PowerShell/Operational
    event_id: 4103, 4104, 4105, 4106

  - name: ForwardedEvents
    tags: [forwarded]
# ====================== Elasticsearch template settings =======================

setup.template.settings:
  index.number_of_shards: 1

# ================================== General ===================================
write_ahead: 2048

# =================================== Kibana ===================================

setup.kibana:

  host: 'https://xxx.xxx.xxx.xxx:xxxx'
  ssl.enabled: true
  username: "username"
  password: "password"

# ================================== Outputs ===================================

  hosts: ['https://xxx.xxx.xxx.xxx:xxxx', 'https://xxx.xxx.xxx.xxx:xxxx', 'https://xxx.xxx.xxx.xxx:xxxx']
  protocol: "https"
  username: "username"
  password: "password"
  index: "index-name"
# ============================= X-Pack Monitoring ==============================
monitoring.enabled: false


setup.template.name: "template-name"
setup.template.name: "template-pattern"
output.elasticsearch.ssl.certificate_authorities: ["path to pem\\elasticsearch-ca.pem"]
setup.ilm.enabled: true
setup.ilm.policy_name: "policy-name"
4

and all log entries end with:

{"log.level":"info","@timestamp":"2024-12-21T06:03:59.329+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"winlogbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":393468,"time":{"ms":250}},"total":{"ticks":1038780,"time":{"ms":297},"value":1038780},"user":{"ticks":645312,"time":{"ms":47}}},"info":{"ephemeral_id":"347f1cb7-99a5-41d5-982a-4410e72c04d6","uptime":{"ms":54600122},"version":"8.16.1"},"memstats":{"gc_next":53790064,"memory_alloc":44754496,"memory_total":88799208520,"rss":105578496},"runtime":{"goroutines":46}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":450,"active":0,"batches":3,"total":450},"read":{"bytes":12155,"errors":3},"write":{"bytes":165398,"latency":{"histogram":{"count":5802,"max":792,"mean":109.5810546875,"median":78,"min":15,"p75":104,"p95":404.5,"p99":592.25,"p999":791.0750000000008,"stddev":117.12300422051013}}}},"pipeline":{"clients":7,"events":{"active":133,"published":414,"total":414},"queue":{"acked":450,"added":{"bytes":1197830,"events":414},"consumed":{"bytes":1359364,"events":450},"filled":{"bytes":372462,"events":133,"pct":0.0415625},"max_bytes":0,"max_events":3200,"removed":{"bytes":1359364,"events":450}}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-12-21T06:04:29.333+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"winlogbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":393703,"time":{"ms":235}},"total":{"ticks":1039218,"time":{"ms":438},"value":1039218},"user":{"ticks":645515,"time":{"ms":203}}},"info":{"ephemeral_id":"347f1cb7-99a5-41d5-982a-4410e72c04d6","uptime":{"ms":54630125},"version":"8.16.1"},"memstats":{"gc_next":54838120,"memory_alloc":41856960,"memory_total":88821422528,"rss":105521152},"runtime":{"goroutines":46}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":473,"active":0,"batches":3,"total":473},"read":{"bytes":12752,"errors":3},"write":{"bytes":162920,"latency":{"histogram":{"count":5805,"max":792,"mean":109.384765625,"median":78,"min":15,"p75":104,"p95":404.5,"p99":592.25,"p999":791.0750000000008,"stddev":116.93512157833}}}},"pipeline":{"clients":7,"events":{"active":94,"published":434,"total":434},"queue":{"acked":473,"added":{"bytes":1298257,"events":434},"consumed":{"bytes":1388400,"events":473},"filled":{"bytes":282319,"events":94,"pct":0.029375},"max_bytes":0,"max_events":3200,"removed":{"bytes":1388400,"events":473}}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-12-21T06:04:59.326+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"winlogbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":394031,"time":{"ms":328}},"total":{"ticks":1039656,"time":{"ms":438},"value":1039656},"user":{"ticks":645625,"time":{"ms":110}}},"info":{"ephemeral_id":"347f1cb7-99a5-41d5-982a-4410e72c04d6","uptime":{"ms":54660119},"version":"8.16.1"},"memstats":{"gc_next":53719600,"memory_alloc":39825336,"memory_total":88845631216,"rss":105521152},"runtime":{"goroutines":46}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":435,"active":0,"batches":3,"total":435},"read":{"bytes":11765,"errors":3},"write":{"bytes":156697,"latency":{"histogram":{"count":5808,"max":792,"mean":109.3271484375,"median":78,"min":15,"p75":103.75,"p95":404.5,"p99":592.25,"p999":791.0750000000008,"stddev":116.95130950191341}}}},"pipeline":{"clients":7,"events":{"active":131,"published":472,"total":472},"queue":{"acked":435,"added":{"bytes":1429793,"events":472},"consumed":{"bytes":1305270,"events":435},"filled":{"bytes":406842,"events":131,"pct":0.0409375},"max_bytes":0,"max_events":3200,"removed":{"bytes":1305270,"events":435}}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-12-21T06:05:29.327+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"winlogbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":394218,"time":{"ms":187}},"total":{"ticks":1040014,"time":{"ms":358},"value":1040014},"user":{"ticks":645796,"time":{"ms":171}}},"info":{"ephemeral_id":"347f1cb7-99a5-41d5-982a-4410e72c04d6","uptime":{"ms":54690120},"version":"8.16.1"},"memstats":{"gc_next":55336880,"memory_alloc":35120736,"memory_total":88865057112,"rss":105209856},"runtime":{"goroutines":46}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":428,"active":0,"batches":3,"total":428},"read":{"bytes":11582,"errors":3},"write":{"bytes":195075,"latency":{"histogram":{"count":5811,"max":792,"mean":109.3271484375,"median":78,"min":15,"p75":103.75,"p95":404.5,"p99":592.25,"p999":791.0750000000008,"stddev":116.95130950191341}}}},"pipeline":{"clients":7,"events":{"active":36,"published":333,"total":333},"queue":{"acked":428,"added":{"bytes":1516022,"events":333},"consumed":{"bytes":1808161,"events":428},"filled":{"bytes":114703,"events":36,"pct":0.01125},"max_bytes":0,"max_events":3200,"removed":{"bytes":1808161,"events":428}}}},"system":{"handles":{"open":-2}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-12-21T06:05:59.324+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"winlogbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":394421,"time":{"ms":203}},"total":{"ticks":1040374,"time":{"ms":360},"value":1040374},"user":{"ticks":645953,"time":{"ms":157}}},"info":{"ephemeral_id":"347f1cb7-99a5-41d5-982a-4410e72c04d6","uptime":{"ms":54720117},"version":"8.16.1"},"memstats":{"gc_next":55336880,"memory_alloc":50128312,"memory_total":88880064688,"rss":105324544},"runtime":{"goroutines":46}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":210,"active":0,"batches":3,"total":210},"read":{"bytes":5913,"errors":3},"write":{"bytes":80814,"latency":{"histogram":{"count":5814,"max":792,"mean":109.3271484375,"median":78,"min":15,"p75":103.75,"p95":404.5,"p99":592.25,"p999":791.0750000000008,"stddev":116.95130950191341}}}},"pipeline":{"clients":7,"events":{"active":119,"published":293,"total":293},"queue":{"acked":210,"added":{"bytes":852813,"events":293},"consumed":{"bytes":618383,"events":210},"filled":{"bytes":349133,"events":119,"pct":0.0371875},"max_bytes":0,"max_events":3200,"removed":{"bytes":618383,"events":210}}}},"system":{"handles":{"open":4}}},"ecs.version":"1.6.0"}}
5

Does it stop printing those nonzero metrics logs? Or does it keep printing those to the log?

Asked another way, is winlogbeat still running long after the "2024-12-21T06:05:59.324" was printed or does it keep printing those nonzero metric messages but you're not seeing anything in Elasticsearch?