Forward Archived .evtx files

Elastic Stack Beats
1

I'm trying to use this winlogbeat config suggested by elastic to forward archived events, however I'm having issues.

It paritially works I do receive events into Elastic however, randomly it will stop sending events and start sending Windows metrics from my agent I'm forwarding the logs with instead.

The desired behavior is for it to stream those logs then stop. However, that is not the experience I'm having.

I'm trying to do this with SOF-ELK (version 8.8.1) with winlogbeat version 8.8.1. I pushed the dasboards before I ingested the logs.

For the logs that do get ingested it doesn't seem like parse them correctly. When I look at the premade dasboard for Winlogbeat there are many empty fields (e.g., username).

2

Can you please share an example of the Windows metrics as well as the exact winlogbeat.yml and command line you're using to run it?

3

It looks identical to to the example file, the only thing I changed was where the elasticsearch server was at on the "output" line.

Command I'm using to iterate logs: (requires CMD prompt)

for /r "C:\Test_logs\" %f in (*.evtx) do .\winlogbeat.exe -e -c .\archive-evtx.yml -E EVTX_FILE="%f"

Windows monitoring that will randomly start instead of job stopping like it should:

{"log.level":"info","@timestamp":"2024-10-11T14:02:32.061-0600","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":145},"message":"Starting metrics logging every 30s","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-10-11T13:51:19.295-0600","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"winlogbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":31,"time":{"ms":31}},"total":{"ticks":62,"time":{"ms":62},"value":62},"user":{"ticks":31,"time":{"ms":31}}},"info":{"ephemeral_id":"3d2cc3fe-e247-4071-853e-a529342130a0","name":"winlogbeat","uptime":{"ms":32640},"version":"8.8.1"},"memstats":{"gc_next":9869064,"memory_alloc":4702000,"memory_sys":24828776,"memory_total":18097616,"rss":41033728},"runtime":{"goroutines":16}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0},"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0},"queue":{"max_events":4096}}},"system":{"cpu":{"cores":32},"handles":{"open":230}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-10-11T13:51:49.286-0600","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"winlogbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":31},"total":{"ticks":62,"value":62},"user":{"ticks":31}},"info":{"ephemeral_id":"3d2cc3fe-e247-4071-853e-a529342130a0","uptime":{"ms":62632},"version":"8.8.1"},"memstats":{"gc_next":9869064,"memory_alloc":4797744,"memory_total":18193360,"rss":41054208},"runtime":{"goroutines":16}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":1,"events":{"active":0}}},"system":{"handles":{"open":-3}}},"ecs.version":"1.6.0"}}

Edit added line that shows "Metrics being started" (First line)

4

Dashboard part of this issue is paritally resolved.

I see why the dashoards aren't working for some reason a majority weren't assoicated with "winlogbeat-*"

I updated the visuilzaitons and those work now. However, the saved queires are giving me this error below even though winlogbeat-* exists...

"Saved object is missing
Could not locate that data view (id: winlogbeat-*), [click here to re-create it](http://192.168.1.5:5601/app/management/kibana/dataViews)

I really wish there was a way to batch convert visiualizations to another Index from the GUI! This has to have been asked for thousands of times!

5

Just following up I provided the information you requested, but haven't heard back.

6

Can you share a screenshot of your dataview that is called winlogbeat-*? This is specifically referring to a dataview, not the indices themselves

7

Here is the screeshot you requested. Also, note I updated to latest version of SOF-ELK which now uses v8.15.2 however, the problem is the same. I'm using the matching Winlogbeat as well.

kibana_dataview
kibana_dataview1444×760 45.3 KB