Log archiving - Difference between ES vs LS

Elastic Stack Beats
1

Hi, I'm brand new to this and learning as I go slowly.

I've setup a ELK stack to store windows event logs, which had been working successfully.

I'm unclear on one point though, at present I am sending direct from winbeatlog to elasticsearch.
Will this archive all the logs it is sent historically or should I have been going winbeatlog to logstash here?

I just need to confirm in case I have missed or confused something and going straight to elastic only "streams" the latest logs for searching, can't seem to find a clear enough answer.

my winbeatlog.yml file is setup as below

winlogbeat.event_logs:

  • name: Application
    ignore_older: 72h
  • name: Security
  • name: System

Hope this makes sense, thanks in advance.

2

You only need to add in Logstash if you want to do additional processing of the data before it gets written to Elasticsearch. Like if you wanted parse something from the event or enrich it.

No matter what output you have configured in Winlogbeat the behavior will be the same. It will start reading from the beginning of the log (or from 72h ago in the case where ignore_older is set), send a batch of events to the output, wait for the output to acknowledge receipt of the events, then Winlogbeat records the last event record number from that batch to the registry_file, then it repeats.

When Winlogbeat is restarted it will resume from the record numbers stored in the registry file.

3

Ok, that sounds good.

As I am trying to use Elasticsearch as the central repository and archive of our Windows server logs, I just wanted to check this is what I had actually achieved.

I'm unable to go back later than 72 hours at present to answer this simple question.... as the index has fallen over (status:red) so I thought I'd take a minute to remove this nagging doubt from my mind before continuing to troubleshoot!

Thanks for the reply and tidying up the question title.

4

During your testing/evaulation of Winlogbeat if you need to reset its state you can stop Winlogbeat, delete the registry file (C:\ProgramData\winlogbeat.winlogbeat.yml), and restart WLB. It will reship everything from the start of the log or point in time determined by ignore_older.

5

thanks again, useful to know :slight_smile: hope I can avoid that as it was working fine right until the last server (of about 40) went on :confused:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.