# Best practice for indexing a daily log

**URL:** <https://discuss.elastic.co/t/best-practice-for-indexing-a-daily-log/118333>\
**Category:** Logstash\
**Created:** [February 4, 2018, 9:12am UTC](https://discuss.elastic.co/t/best-practice-for-indexing-a-daily-log/118333 "2018-02-04T09:12:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![David\_Gidony](https://avatars.discourse-cdn.com/v4/letter/d/ed655f/32.png) [@David\_Gidony](https://discuss.elastic.co/u/David_Gidony)\
**Post date:** [February 4, 2018, 9:12am UTC](https://discuss.elastic.co/t/best-practice-for-indexing-a-daily-log/118333/1 "2018-02-04T09:12:02Z")

</div>

hi,  
i have a few log files that are created daily,  
i wanted to know what is the best way to ingest them,  
should i create a new index with the current date for each log file, or just add them all to one big index.  
for example:  
i have dca.log.16.01.2018.log  
dca.log.17.01.2018.log  
dca.log.18.01.2018.log  
dca.log.19.01.2018.log

should i create a new index every day ? or just create a index called "dca logs" and just add everything there ?  
is there a limit for records in one index ? is there any advantage for splitting indexes?

what is the right way ?  
i'm a noob in elk , and still trying to figure these thing up.

thanks  
David

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 4, 2018, 9:13am UTC](https://discuss.elastic.co/t/best-practice-for-indexing-a-daily-log/118333/2 "2018-02-04T09:13:56Z")

</div>

How large are your logs and how long do you want to keep them in the cluster?

Using time-based indices is great for managing retention, and considered best practice for log data. You do however not want a lot of small indices and shards as that can be very inefficient. It may therefore be good to use e.g. monthly indices instead or daily if your daily data volumes are low. You can find further guidance in [this blog post](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 4, 2018, 9:20am UTC](https://discuss.elastic.co/t/best-practice-for-indexing-a-daily-log/118333/3 "2018-03-04T09:20:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.