Best Practices for Efficient and Effective Log Storage and Retrieval with Elasticsearch and Logstash?

Elastic Stack Elasticsearch
4

Create either a data stream or a set of time-based indices.

A datastream will create a series of backing indices each containing data indexed during a specific period. You index into an alias that writes to the latest index and can query all indices.

Another way is to have Logstash create time-based indices. You can do this by specifying the index name as follows:

elasticsearch {
  index => "logstash-%{+YYYY.MM.dd}"
}

Irrespective of the method chosen, you then make sure in Logstash that you store the data you want to be able to filter on in the actual events, e.g.:

{
  "testlevel": "x",
  "casename": "y",
  "start_time": "2023-03-17T03:17:00Z",
  ...
}
1 Like