Repository for logs

Elastic Stack Logstash
1

I receive about 2-3GB of weekly machine (software application) logs to troubleshoot. I'm in a limited budget and I am trying to learn Elasticsearch, Logstash, Filebeat how to store and archive these logs in a common repository and purge older logs based a date rules.

  1. I'm trying to create a model (e.g. script) that allows to learn about the string patterns from these logs. The goal is to find root cause of the software application problem.
    I will appreciate your feedbacks. Thanks.
2

So you want to do automatic root cause analysis?

3

Hi @warkolm. Automatic Root cause analysis would be nice. But I am first concerned of the disk or repo for the logs since I receive about 2-3GB weekly.

4

What's your concern though?

5

the cost of the disk space as I'm concerned since I receive 2-3 GB log file size weekly. How can I reduce the cost? Ideas?

6

That's not a lot though, so unless you budget is a few dollars a month then you might be worrying about the wrong thing.

If it's a big problem, then look at using _rollup, or only keeping a few days of data around.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.