# Best practice in elasticsearch idnex

**URL:** <https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815>\
**Category:** Elasticsearch\
**Created:** [August 18, 2021, 12:02pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815 "2021-08-18T12:02:01Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![naaviin](https://avatars.discourse-cdn.com/v4/letter/n/bcef8e/32.png) [@naaviin](https://discuss.elastic.co/u/naaviin)\
**Post date:** [August 18, 2021, 12:02pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815/1 "2021-08-18T12:02:01Z")

</div>

I have few log with some same patter and some different patterh.

Log name (same pattern)

```auto
a.log
b.log

```

Log name (different pattern)

```auto
c.log
d.log

```

Environment name

```auto
env1
env2
env3

```

Is this good practice for index  
--MMYYDD

example  
env1-a.log-20210818

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 18, 2021, 12:46pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815/2 "2021-08-18T12:46:22Z")

</div>

Having lots of small indices is very very inefficient and scales badly. It looks like your scheme could produce lots of small indices so I would not recommend it. What is wrong with a single time-based index, e.g. managed using rollover?

---

<div class="post-metadata">

**Author:** ![naaviin](https://avatars.discourse-cdn.com/v4/letter/n/bcef8e/32.png) [@naaviin](https://discuss.elastic.co/u/naaviin)\
**Post date:** [August 18, 2021, 3:02pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815/3 "2021-08-18T15:02:36Z")

</div>

can share details. then during table view in kibana how i can only display for a single log. how you will expect the index patter will be. i worried if all log use 1 index can cause slow as there are 15 logs total for 1 environment

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 18, 2021, 3:10pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815/4 "2021-08-18T15:10:06Z")

</div>

The general recommendation is to aim for a shard size of at least a few GB and a few tens of GB is very common. You can still have the data you used to create index names in the actual events so that you can filter them in Kibana.

---

<div class="post-metadata">

**Author:** ![naaviin](https://avatars.discourse-cdn.com/v4/letter/n/bcef8e/32.png) [@naaviin](https://discuss.elastic.co/u/naaviin)\
**Post date:** [August 18, 2021, 3:19pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815/5 "2021-08-18T15:19:51Z")

</div>

in my log which i pss i didnt have the log name in it. if this the case how i can filter only a.log. is there a way to append log name so i know this belongs to which logs

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 18, 2021, 3:20pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815/6 "2021-08-18T15:20:52Z")

</div>

How are you shipping the log data? Filebeat?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 15, 2021, 3:21pm UTC](https://discuss.elastic.co/t/best-practice-in-elasticsearch-idnex/281815/7 "2021-09-15T15:21:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.