# Best practice wanted for huge number of index time serial data **URL:** \ **Category:** Elasticsearch\ **Created:** [July 14, 2014, 1:40am UTC](https://discuss.elastic.co/t/best-practice-wanted-for-huge-number-of-index-time-serial-data/18636 "2014-07-14T01:40:52Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![limac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limac/32/3981_2.png) [@limac](https://discuss.elastic.co/u/limac)\ **Post date:** [July 14, 2014, 1:40am UTC](https://discuss.elastic.co/t/best-practice-wanted-for-huge-number-of-index-time-serial-data/18636/1 "2014-07-14T01:40:52Z")
Hi folks, I am trying to index a huge number of time serial data. The total number will be 5k docs for one second which will continue for several months. I also need to search these data, but only inside a very small time rage, maybe one hour. Is there any best practice for this kind of use case? Thanks! Alan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 14, 2014, 2:02am UTC](https://discuss.elastic.co/t/best-practice-wanted-for-huge-number-of-index-time-serial-data/18636/2 "2014-07-14T02:02:33Z")
This is pretty standard for logstash type data. Use daily indexes, don't use TTL. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 14 July 2014 11:40, LiMac [cnwangyong@gmail.com](mailto:cnwangyong@gmail.com) wrote: > Hi folks, > > I am trying to index a huge number of time serial data. The total number > will be 5k docs for one second which will continue for several months. I > also need to search these data, but only inside a very small time rage, > maybe one hour. Is there any best practice for this kind of use case? > > Thanks! > > Alan > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt\_oY3ZSyqNZoBpzw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt_oY3ZSyqNZoBpzw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![limac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limac/32/3981_2.png) [@limac](https://discuss.elastic.co/u/limac)\ **Post date:** [July 17, 2014, 12:27am UTC](https://discuss.elastic.co/t/best-practice-wanted-for-huge-number-of-index-time-serial-data/18636/3 "2014-07-17T00:27:27Z")
Thank you Mark, if I use daily index, I have to specify multiple indexes based on the time range. That will make my service a little more complicate. So I am wondering, even if I put all data in one huge index, as long as I limit the time range in my query, it looks like es will locate the data as quickly as I do it in a much smaller daily index, cause es will not need to search through the whole index, just need to locate the data first by the time range specified in the query. Is that true? Alan From: [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [[mailto:elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)] On Behalf Of Mark Walkom Sent: 2014年7月14日 10:03 To: [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) Subject: Re: best practice wanted for huge number of index time serial data This is pretty standard for logstash type data. Use daily indexes, don't use TTL. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) [mailto:markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) [http://www.campaignmonitor.com](http://www.campaignmonitor.com) On 14 July 2014 11:40, LiMac \<[cnwangyong@gmail.com](mailto:cnwangyong@gmail.com) [mailto:cnwangyong@gmail.com](mailto:cnwangyong@gmail.com) \> wrote: Hi folks, I am trying to index a huge number of time serial data. The total number will be 5k docs for one second which will continue for several months. I also need to search these data, but only inside a very small time rage, maybe one hour. Is there any best practice for this kind of use case? Thanks! Alan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) . To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com?utm_medium=email&utm_source=footer) . For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) . To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt\_oY3ZSyqNZoBpzw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt_oY3ZSyqNZoBpzw%40mail.gmail.com) [https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt\_oY3ZSyqNZoBpzw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt_oY3ZSyqNZoBpzw%40mail.gmail.com?utm_medium=email&utm_source=footer) . For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/010601cfa155%24e55579b0%24b0006d10%24%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/010601cfa155%24e55579b0%24b0006d10%24%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:15am UTC](https://discuss.elastic.co/t/best-practice-wanted-for-huge-number-of-index-time-serial-data/18636/4 "2017-07-06T01:15:17Z")