Winlogbeat delayed data shipping

Hi there,

I'm collecting security register data from Active Directory dedicated Windows Server 2012 host, that pass via Logstash and finally land in ES. After the week-end I've a found an huge delay of 15 hours between data sent by winlogbeat and the current time when I search for them in Kibana. Beat and Logstash logs don't show any error and the sysadmin of the Domain controller says that the AD machine is okay. I've tried to write data that pass in Logstash in a file and they are actually related to 15 hours ago events.
What can I do more to do troubleshooting and align the time?
Thank you very much.

Which version are you using?

How many events per second is Winlogbeat processing approximately?

Do you see any WARN on ERROR in Winlogbeat log?

Such a huge time difference would usually indicate a timezone confusion between Beats and Kibana.

Thank you very much for the answer.
I'm using 6.2.4 version of winlogbeat and 6.7.1 version for Logstash. Approximately winlogebat publishes 3000 events per second. The strange thing is that I'm reading 2 registry of event viewer but just one is disaligned.

2019-05-26T06:02:18.981+0200	ERROR	logstash/async.go:235	Failed to publish events caused by: write tcp> wsasend: An existing connection was forcibly closed by the remote host.

this is an ERROR log but is such "normal" beacause I see it since months.

As you said, I don't think that error is an issue, as long as you receive it just from time to time. Logstash expires connections after a while, and the Beat can only find out it's closed once it tries to write. Winlogbeat will create a new connection immediately without any event being lost.

Is it possible that the Event Log that is showing this delay is generating more than this 3000 events per second?

Yes.I think that could be possible. The data volume for that registry is about 20 mln of records per day. I wish to know if there is any "tune tip" to try aligning the data, because now I've reached 17 hours of delay. I hope that this huge volume is transitory.

Thank you for the interest.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.