# Creating a single huge index or multiple indexes rolled out with a date

**URL:** <https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210>\
**Category:** Elasticsearch\
**Created:** [January 11, 2017, 11:41am UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210 "2017-01-11T11:41:32Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sari.reach](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@sari.reach](https://discuss.elastic.co/u/sari.reach)\
**Post date:** [January 11, 2017, 11:41am UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210/1 "2017-01-11T11:41:32Z")

</div>

I am using Elastic Search Java APIs to create the index, write to index and search for data from these indexes. To attain high availability and archiving. My index may acquire a size of 10mb on daily basis. Is it advisable to create a single index or to create index on a daily basis with a date update on weekly basis. If i have the same index with different dates. How difficult it is to search using elastic search API's using these indexes

I am using Elastic search 1.7.3 currently and i may switch to 2.3 going further.

---

<div class="post-metadata">

**Author:** ![xavierfacq](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xavierfacq/32/8744_2.png) [@xavierfacq](https://discuss.elastic.co/u/xavierfacq)\
**Post date:** [January 11, 2017, 12:34pm UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210/2 "2017-01-11T12:34:46Z")

</div>

> [@sari.reach](#):
>
> If i have the same index with different dates. How difficult it is to search using Elasticsearch API's using these indexes

You can set the same Alias on your indices to search over all : [Index Aliases | Elasticsearch Guide [5.1] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/5.1/indices-aliases.html)

Depending on your datasets, you can imagine to create an index per day, week or month.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 11, 2017, 10:52pm UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210/3 "2017-01-11T22:52:39Z")

</div>

Don't use a monolithic index.  
Do use time based, given the size of the data, a weekly one with only a small shard count would be ideal.

> [@sari.reach](#):
>
> I am using Elastic search 1.7.3 currently and i may switch to 2.3 going further.

Why not 5.1?

---

<div class="post-metadata">

**Author:** ![sari.reach](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@sari.reach](https://discuss.elastic.co/u/sari.reach)\
**Post date:** [January 12, 2017, 4:08am UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210/4 "2017-01-12T04:08:00Z")

</div>

5.1 doesn't support AWS deployment. Only 2.3 supports AWS, so only planning to upgrade to 2.3 in next 2 weeks. Does 5.1 supports AWS?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 12, 2017, 4:30am UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210/5 "2017-01-12T04:30:24Z")

</div>

What do you mean support AWS?  
You can run any ES version on an EC2 instance.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [January 12, 2017, 5:29am UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210/6 "2017-01-12T05:29:48Z")

</div>

@sari.reach I think you're phrasing it backwards. You said "5.1 doesn't support AWS deployment," but what you should have said was "AWS ES cannot deploy version 5.1 yet. AWS ES is only at version 2.3 of Elasticsearch."

What @warkolm is talking about is running your _own_ Elasticsearch instance, self-managed, on an EC2 instance you deploy, rather than the AWS ES managed solution. There are reasons this is good, as there are other limitations to the AWS ES service besides just being behind several versions.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 9, 2017, 5:30am UTC](https://discuss.elastic.co/t/creating-a-single-huge-index-or-multiple-indexes-rolled-out-with-a-date/71210/7 "2017-02-09T05:30:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
