Winlogbeat stops working after a while with errors

Elastic Stack Beats
1

Hi, I just install winlogbeat in a Windows 2003 Server and sent the logs to Logstash 2.3.1.
This is my Winlogbeat conf:
output.logstash: hosts: ["192.168.70.110:5044"]
(complete file: http://pastebin.com/KdTTPEu2)

My Logstash conf:
input { beats { port => 5044 } } output { file { path => "/data/winlogbeat-%{+YYYY.MM.dd}.log" } }
And it worked fine for a while, and suddenly stopped and said this:
2016/11/23 17:47:09.829198 outputs.go:100: ERR failed to initialize logstash plugin as output: missing required field accessing 'output.logstash.hosts' 2016/11/23 17:47:09.829198 beat.go:285: CRIT Exiting: error initializing publisher: missing required field accessing 'output.logstash.hosts' Exiting: error initializing publisher: missing required field accessing 'output.logstash.hosts'
(the whole file here: http://pastebin.com/f25ty2jb)

This is on the winlogbeat side. In the logstash side, I don't see anything weird.

Any idea on what is the problem?
Thanks!

Sacha.

2

I don't see any obvious issues with your configuration file. I cannot explain why repeating the same config test over and over again would cause different results. Can you strip down the config file to just the bare minimum, make sure that the file is plain ascii encoding, and not open in any editors. Then re-run the config test with .\winlogbeat.exe -c winlogbeat.yml -e -d "*" -configtest.

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: Security
  - name: System

output.logstash:
  hosts: ["192.168.70.110:5044"]

logging.level: debug

If that passes reliably then we can move on and work on debugging the crash that was in the log file you posted.

3

@andrewkroh, thanks for your answer.
I just did it. When I ran it with configtest it ended with a "Config OK".
So far, so good.
I ran it again, just with .\winlogbeat.exe -c winlogbeat.yml, and then it sent one log per day starting on Nov 21st, 2014 until today.
So far, so good...
The weird part, is that before, even it send it a lot less files (65MB before vs 375MB now), at that time it sent starting on Aug 23th, 2013 until today, skipping a lot of days in between.

Last thing: My understanding is that it should send Application log for the last 72 hours and not older than that. That would explain why older logs are very light (5KB), and latest ones a lot more heavy (50MB+). If that is true, I'd expect big logs would be from the last 3 days... but they are big fro the last 7 days (Nov 17, 18, 19, 20, 21, 22 and 23rd). Is this an expected behavior?

4

Regarding ignore_older, as configured that only applies to the events coming from the Application event log. You do need to specify it on an individual log basis (see example below). This could explain what you are seeing.

This setting configures a filter in the query that goes to the Windows API and basically says only give me an event if it occurred with the prior 72h. This is mostly useful for the first time you run Winlogbeat to prevent months (or years ) of old logs from being ingested, but after the first run Winlogbeat have stored it's state (current read offset for each log) to disk. So if you restart it will resume from the last known event (and the ignore older filter will also be applied).

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: Security
    ignore_older: 72h
  - name: System
    ignore_older: 72h
5

Got it, thanks. I'm still curious why downloaded 7 "heavy" days instead of 3 (72 hours).
I noticed that the "groking" is made on the winlogbeat side, not in logstash nor ES. Is there any way to rename the event_data.paramXX fields? Or maybe they are called like that because it means different stuff in different messages?

6

See Generically named event_data.paramN on Windows XP and 2003 - #2 by andrewkroh

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.