I'm collecting security register data from Active Directory dedicated Windows Server 2012 host, that pass via Logstash and finally land in ES. After the week-end I've a found an huge delay of 15 hours between data sent by winlogbeat and the current time when I search for them in Kibana. Beat and Logstash logs don't show any error and the sysadmin of the Domain controller says that the AD machine is okay. I've tried to write data that pass in Logstash in a file and they are actually related to 15 hours ago events.
What can I do more to do troubleshooting and align the time?
Thank you very much.
Thank you very much for the answer.
I'm using 6.2.4 version of winlogbeat and 6.7.1 version for Logstash. Approximately winlogebat publishes 3000 events per second. The strange thing is that I'm reading 2 registry of event viewer but just one is disaligned.
2019-05-26T06:02:18.981+0200 ERROR logstash/async.go:235 Failed to publish events caused by: write tcp 10.160.0.12:59890->10.160.5.130:5051: wsasend: An existing connection was forcibly closed by the remote host.
this is an ERROR log but is such "normal" beacause I see it since months.
As you said, I don't think that error is an issue, as long as you receive it just from time to time. Logstash expires connections after a while, and the Beat can only find out it's closed once it tries to write. Winlogbeat will create a new connection immediately without any event being lost.
Is it possible that the Event Log that is showing this delay is generating more than this 3000 events per second?
Yes.I think that could be possible. The data volume for that registry is about 20 mln of records per day. I wish to know if there is any "tune tip" to try aligning the data, because now I've reached 17 hours of delay. I hope that this huge volume is transitory.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.