# Best Practises to handle an index of of 1.2TiB **URL:** \ **Category:** Elasticsearch\ **Created:** [February 1, 2021, 1:08pm UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839 "2021-02-01T13:08:36Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![carmine.fabrizio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carmine.fabrizio/32/71788_2.png) [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)\ **Post date:** [February 1, 2021, 1:08pm UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/1 "2021-02-01T13:08:36Z")
Hi there, I have an index of 1.2TiB, split into 9 shards of 200GiB each, also I do have a lifecycle policy that will roll the shards in HOT/WARM/COLD phases. Are there any best practices that I can introduce in order to facilitate the search? Thanks ---
**Author:** ![Vadim\_Yakhin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadim_yakhin/32/56723_2.png) [@Vadim\_Yakhin](https://discuss.elastic.co/u/Vadim_Yakhin)\ **Post date:** [February 1, 2021, 6:02pm UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/2 "2021-02-01T18:02:21Z")
Hi Carmine, Just to clarify, have you indexed the data through the [Elastic Enterprise Search](https://www.elastic.co/enterprise-search) (which includes App Search and Workplace Search), or are you using [elasticsearch](https://www.elastic.co/elasticsearch/)? If it's elasticsearch, [Elastic Stack subforum](https://discuss.elastic.co/c/elastic-stack/) would be a better place to ask this question. ---
**Author:** ![carmine.fabrizio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carmine.fabrizio/32/71788_2.png) [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)\ **Post date:** [February 2, 2021, 11:17am UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/3 "2021-02-02T11:17:13Z")
Hello @Vadim_Yakhin , no have not used Elastic Enterprise Search, using normal elasticsearch. Cannot create any topic into [Elastic Stack subforum](https://discuss.elastic.co/c/elastic-stack/) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 2, 2021, 11:19am UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/4 "2021-02-02T11:19:04Z")
Are you using time based indices? Can you elaborate on the use case? ---
**Author:** ![carmine.fabrizio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carmine.fabrizio/32/71788_2.png) [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)\ **Post date:** [February 2, 2021, 11:24am UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/5 "2021-02-02T11:24:54Z")
not using time based indices, the major issue is on search ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/a/ba48e7fe9dc7f1f6e1cc263b465b473edc12d543.png) and I am not sure where the bottle neck is ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 2, 2021, 11:28am UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/6 "2021-02-02T11:28:24Z")
If you do not have time based indices ILM and zoning does not really apply. What is the issue with searching? It looks like load has increased but latencies are seemingly low. ---
**Author:** ![carmine.fabrizio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carmine.fabrizio/32/71788_2.png) [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)\ **Post date:** [February 2, 2021, 12:09pm UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/7 "2021-02-02T12:09:58Z")
there is a misunderstanding here, I'll clarify by sharing some configuration. this is the configuration that I'm passing when creating an ILM ``` PUT _ilm/policy/test-logs { "policy": { "phases": { "hot": { "min_age": "0ms", "actions": { "rollover": { "max_age": "1d", "max_size": "200gb" }, "set_priority": { "priority": 100 } } }, "warm": { "actions": { "allocate": { "include": {}, "exclude": {}, "require": { "data": "warm" } }, "forcemerge": { "max_num_segments": 1 }, "set_priority": { "priority": 50 }, "shrink": { "number_of_shards": 9 } } }, "cold": { "min_age": "14d", "actions": { "allocate": { "include": {}, "exclude": {}, "require": { "data": "cold" } }, "freeze": {}, "set_priority": { "priority": 20 } } }, "delete": { "min_age": "60d", "actions": { "delete": {} } } } } } ``` then I'll pass the template ``` { "index": { "lifecycle": { "name": "test-logs", "rollover_alias": "test-logs" }, "routing": { "allocation": { "require": { "data": "hot" } } }, "refresh_interval": "5s", "number_of_shards": "9", "number_of_replicas": "0" } } ``` and this is what I have configured on the logstash as output ``` output { elasticsearch { hosts => ["elasticxxxxx.local"] ilm_enabled => true ilm_rollover_alias => "test-logs" template_name => "test-logs" manage_template => false index => "test-logs" ilm_pattern => "000001" ilm_policy => "test-logs" } } ``` normal behavior ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/7/67b67a04758dcf8a07b878a77b913a6549c35be1.png) last 12h during office time ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 2, 2021, 12:10pm UTC](https://discuss.elastic.co/t/best-practises-to-handle-an-index-of-of-1-2tib/262839/8 "2021-03-02T12:10:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.