# Elastic best Practice

**URL:** <https://discuss.elastic.co/t/elastic-best-practice/312671>\
**Category:** Elasticsearch\
**Created:** [August 23, 2022, 5:22am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671 "2022-08-23T05:22:29Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\
**Post date:** [August 23, 2022, 5:22am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/1 "2022-08-23T05:22:29Z")

</div>

Hello, I am indexing 1mil logs per 15 minutes in one index and I need about 10 000 of them to visualize in Grafana and make reports on it.  
Is it good practice to separate these 10 000/15 mins to another index? Will searching be quicker when I rotate it daily and have a smaller shard size then?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 23, 2022, 5:53am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/2 "2022-08-23T05:53:32Z")

</div>

> [@vnovotny98](#):
>
> 10 000 of them

What is them here?

---

<div class="post-metadata">

**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\
**Post date:** [August 23, 2022, 6:41am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/3 "2022-08-23T06:41:48Z")

</div>

I am indexing 1mil and need to visualize and use for reporting +- 10k of logs. But I must collect everything. Should I maybe duplicate these 10k logs to another index or split them to another index? Or it doesn't matter that I have 1mil logs and search only for 10k of them.  
1 mil is all application logs and about 10K of them is from trades that I need to visualize.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 23, 2022, 6:45am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/4 "2022-08-23T06:45:35Z")

</div>

Are the logs the same format for both?

---

<div class="post-metadata">

**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\
**Post date:** [August 23, 2022, 6:49am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/5 "2022-08-23T06:49:49Z")

</div>

Yes they are. Logs are pushed from app by Logback Elasticsearch Appender

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 23, 2022, 6:51am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/6 "2022-08-23T06:51:11Z")

</div>

Unless there's no easy way to filter between each log type, I wouldn't worry too much about splitting them out with that sort of volume.

---

<div class="post-metadata">

**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\
**Post date:** [August 23, 2022, 6:54am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/7 "2022-08-23T06:54:22Z")

</div>

I maybe misunderstood. I can have easy filter for that. When LogType= TrxPersist, it means that it is transaction. I can filter on that in Logstash. But I don't know if it will be better when these transactions will be in one index, or they will be in index with many other logs that I will not search for unless some debug or problem will start.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 23, 2022, 7:03am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/8 "2022-08-23T07:03:15Z")

</div>

Elasticsearch should be find with this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 20, 2022, 7:03am UTC](https://discuss.elastic.co/t/elastic-best-practice/312671/9 "2022-09-20T07:03:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
