# Guidance on Using Multiple Indexes vs One Index for Time Series Data from Multiple Sources

**URL:** <https://discuss.elastic.co/t/guidance-on-using-multiple-indexes-vs-one-index-for-time-series-data-from-multiple-sources/32729>\
**Category:** Elasticsearch\
**Created:** [October 21, 2015, 10:51pm UTC](https://discuss.elastic.co/t/guidance-on-using-multiple-indexes-vs-one-index-for-time-series-data-from-multiple-sources/32729 "2015-10-21T22:51:17Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [October 23, 2015, 5:55pm UTC](https://discuss.elastic.co/t/guidance-on-using-multiple-indexes-vs-one-index-for-time-series-data-from-multiple-sources/32729/4 "2015-10-23T17:55:52Z")

</div>

> [@sgt\_b2002](#):
>
> Would there be any drawback to having a single daily index for all data instead of having these per data source indexes?

Like @otisg says, types might be the right thing here. Its what they are for. Each has a fixed cost in heap and cluster state maintenance time for types can help here. The drawbacks come around scoring (IDF information is shared), aggregations (types can hide sparseness and make the doc values less efficient), and types (if two devices send data with the same name but a different type then you'll have a bad time). If you can live with these things then I think you should look at types.

Sparseness is the biggest problem I can think of - I don't know the doc value data structures super well but think of it like this: they pick the number of bits they use to represent integers for an entire segment. If you have two devices, one that only stores data in the 0-255 range and another that stores in the 0-64k range and you mix them into one segment then the whole segment will need two bytes per integer rather than the one. But if you only use these value for searching (not sorting or scripts or aggregation) then this shouldn't come up.

> [@sgt\_b2002](#):
>
> What about IO? Would using one big index split across the 4 nodes vs multiple, smaller indexes make any difference in that regard? We're not using SSDs so IO hurts sometimes.

You might have some success with the total\_shards\_per\_node setting - its a bit touchy because its a hard limit, but it could be useful to make sure that the shards you are actively writing to are more evenly spread out.

In general you'll get more IO load on indexing with one large index than four smaller indexes. But your milage may vary. Its complex.

---

_[View the full topic](https://discuss.elastic.co/t/guidance-on-using-multiple-indexes-vs-one-index-for-time-series-data-from-multiple-sources/32729)._
