Working with logs

We're building a logging system for our project. We want to see almost everything as a log like Docker, application console etc. Since we're growing up, we do not want to make mistake about logs(wrong scheme, indexing, querying etc.)

There are few questions in my mind,

  1. After some times, log file size will grow up and be a problem. What should we do about it?
  2. We want to log user actions like when they purchase something or their message logs etc. For that purpose, what should we do? Should we save their message logs with timestamp to Elasicearch?
  3. We want to analyze everything. Should we use timestamp for every document?

I'd like to hear your ideas. Thanks!

  1. Use [ILM]ILM: Manage the index lifecycle | Elasticsearch Reference [7.11] | Elastic).
  2. Yes, that's a log :slight_smile:
  3. If it's time based data then 100% you should

If we're being honest, you will make a mistake. There's no way to make something completely, 100% correct. You're better off making your system and process resilient to changes.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.