Best practice in elasticsearch idnex

Elastic Stack Elasticsearch
1

I have few log with some same patter and some different patterh.

Log name (same pattern)

a.log
b.log

Log name (different pattern)

c.log
d.log

Environment name

env1
env2
env3

Is this good practice for index
--MMYYDD

example
env1-a.log-20210818

2

Having lots of small indices is very very inefficient and scales badly. It looks like your scheme could produce lots of small indices so I would not recommend it. What is wrong with a single time-based index, e.g. managed using rollover?

3

can share details. then during table view in kibana how i can only display for a single log. how you will expect the index patter will be. i worried if all log use 1 index can cause slow as there are 15 logs total for 1 environment

4

The general recommendation is to aim for a shard size of at least a few GB and a few tens of GB is very common. You can still have the data you used to create index names in the actual events so that you can filter them in Kibana.

5

in my log which i pss i didnt have the log name in it. if this the case how i can filter only a.log. is there a way to append log name so i know this belongs to which logs

6

How are you shipping the log data? Filebeat?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.