# Create new index or use the existing for small but different purpose source logs?

**URL:** https://discuss.elastic.co/t/create-new-index-or-use-the-existing-for-small-but-different-purpose-source-logs/63838
**Category:** Elasticsearch
**Created:** [October 25, 2016, 7:11am UTC](https://discuss.elastic.co/t/create-new-index-or-use-the-existing-for-small-but-different-purpose-source-logs/63838 "2016-10-25T07:11:48Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![zpp](https://avatars.discourse-cdn.com/v4/letter/z/54ee81/32.png) [@zpp](https://discuss.elastic.co/u/zpp)
#### Post date: [October 25, 2016, 7:11am UTC](https://discuss.elastic.co/t/create-new-index-or-use-the-existing-for-small-but-different-purpose-source-logs/63838/1 "2016-10-25T07:11:48Z")

</div>

I have a system, which has quite a number of sub-systems in it, each sub-system will produce its own log files, whose sizes ranges from a few MB, a few hundred MB to a few GB. These sub systems don't really have a lot of common fields (minimal or even no), in this case, should I put them under the same index or put them under different indexes? (putting them in different indexes =\> resulting in a lot of small shards in the cluster, is this recommended?)

---

<div class="post-metadata">

### Author: ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)
#### Post date: [October 25, 2016, 9:14am UTC](https://discuss.elastic.co/t/create-new-index-or-use-the-existing-for-small-but-different-purpose-source-logs/63838/2 "2016-10-25T09:14:18Z")

</div>

Without further input as to what query volume you expect and what your data size will be in total my first advise would be to model your index to be in line with the types of queries you'd expect: If those logs are truly distinct and you will ask different questions to each I'd also put them into different indexes.

For more general input on sizing elasticsearch see:

> **[Sizing Elasticsearch
	  	 | Elastic](https://www.elastic.co/blog/found-sizing-elasticsearch)**
>
> We're often asked 'How big a cluster do I need?', and it's usually hard to be more specific than 'Well, it depends!'. There are so many variables, where knowledge about your application's specific workload and your performance expectations are just...

For more input on your specific question on whether having many shards in one cluster is an issue see here:

[https://www.elastic.co/guide/en/elasticsearch/guide/current/kagillion-shards.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/kagillion-shards.html)

Hope this helps,  
Isabel

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:10pm UTC](https://discuss.elastic.co/t/create-new-index-or-use-the-existing-for-small-but-different-purpose-source-logs/63838/3 "2017-07-05T22:10:04Z")

</div>


