# Index Size Increase? **URL:** \ **Category:** Kibana\ **Created:** [June 30, 2019, 5:34pm UTC](https://discuss.elastic.co/t/index-size-increase/188202 "2019-06-30T17:34:56Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 5:34pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/1 "2019-06-30T17:34:56Z")
I was wondering if there is a way to increase the size of the index for Kibana. I have one index that is 2,181,430,251 and it started to error out saying that I was at max size. I want to keep the size going but not sure how. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 30, 2019, 5:36pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/2 "2019-06-30T17:36:53Z")
Lucene has a strict document limit per shard around 2 billion, so you would need to add shards to increase the size. This may require reindexing. Which version are you on? ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 7:37pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/3 "2019-06-30T19:37:29Z")
I am running the latest version of everything. I am currently at 2,181,xxx,xxx records. How can I add more shards? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 30, 2019, 7:39pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/4 "2019-06-30T19:39:48Z")
Depending on how and when your index was created, you may be able to use the [split index API](https://www.elastic.co/guide/en/elasticsearch/reference/7.2/indices-split-index.html). If that is not an option you will need to [reindex your data](https://www.elastic.co/guide/en/elasticsearch/reference/7.2/docs-reindex.html) into an index with a higher number of shards. ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 7:40pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/5 "2019-06-30T19:40:26Z")
Well I have about 250 billion records so I will need to maximize my shards. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 30, 2019, 7:41pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/6 "2019-06-30T19:41:34Z")
How large are your documents? How many indices and shards do you have at the moment? What is the size and specification of your cluster? ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 7:43pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/7 "2019-06-30T19:43:50Z")
Great questions my documents range from 12mbs to 5kb and I have roughly 300,000 files that equals 113gb. I am running everything standard so I have no adjusted any shards or anything. I have total of 16TB of space for this. ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 7:47pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/8 "2019-06-30T19:47:03Z")
I ran GET \_cat/shards and it came back with this: XXXXXXX\_2 0 p STARTED 70722857 12.9gb X.X.x.x SOMENAME XXXXXXX\_2 0 r UNASSIGNED XXXX 0 p STARTED 2147483519 300.7gb x.x.x.x SOMENAME XXXX 0 r UNASSIGNED .kibana\_task\_manager 0 p STARTED 2 29.9kb x.x.x. SOMENAME .kibana\_1 0 p STARTED 13 55.7kb x.x.x.x SOMENAME ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 7:48pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/9 "2019-06-30T19:48:52Z")
System specs are Proc: i7 Ram: 32 Hard Drive: 16TB + 1tb ssd ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 30, 2019, 7:53pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/10 "2019-06-30T19:53:34Z")
If 2 billion documents take up 300GB, 250 billion is may require up to over 37TB. I also suspect you might run into issues with heap pressure. I therefore suspect you will need a larger cluster to handle this. You may find [this webinar](https://www.elastic.co/webinars/optimizing-storage-efficiency-in-elasticsearch) useful. ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 7:56pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/11 "2019-06-30T19:56:31Z")
ok back to the drawing board. What happens if I create multiple "index" can I merge them all together? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 30, 2019, 7:59pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/12 "2019-06-30T19:59:22Z")
No, but an index can have a lot of shards and you can query across multiple indices, so having a reasonable number of shards should not be a problem. Even though it depends on your data I would recommend keeping shards between 50GB and 100GB in size unless you verify that querying speed and relocation of shards work well at larger sizes. ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [June 30, 2019, 8:03pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/13 "2019-06-30T20:03:43Z")
Ok perfect. Thanks so much for the help I am sure that you will be seeing me around some. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 28, 2019, 8:03pm UTC](https://discuss.elastic.co/t/index-size-increase/188202/14 "2019-07-28T20:03:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.