# ES indexing strategy

**URL:** <https://discuss.elastic.co/t/es-indexing-strategy/58413>\
**Category:** Elasticsearch\
**Created:** [August 19, 2016, 12:18am UTC](https://discuss.elastic.co/t/es-indexing-strategy/58413 "2016-08-19T00:18:37Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![cgswong](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cgswong/32/3395_2.png) [@cgswong](https://discuss.elastic.co/u/cgswong)\
**Post date:** [August 19, 2016, 12:18am UTC](https://discuss.elastic.co/t/es-indexing-strategy/58413/1 "2016-08-19T00:18:37Z")

</div>

Hi all! I'm curious as to how others are doing, or would recommend doing indexing. Specifically, we have several indexes based on document structure such as for syslog, audit logs (two here since the document structures are different), Apache logs, jvm logs, tomcat logs, app logs, etc.

So we've got about 9 indexes (here). I'm thinking maybe we could use less indexes and use more types for some of the documents instead. Thoughts? How are you doing indexing?

Appreciate any feedback.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 19, 2016, 12:20am UTC](https://discuss.elastic.co/t/es-indexing-strategy/58413/2 "2016-08-19T00:20:25Z")

</div>

> [@cgswong](#):
>
> So we've got about 9 indexes (here). I'm thinking maybe we could use less indexes and use more types for some of the documents instead.

Why are you thinking that?

---

<div class="post-metadata">

**Author:** ![cgswong](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cgswong/32/3395_2.png) [@cgswong](https://discuss.elastic.co/u/cgswong)\
**Post date:** [August 19, 2016, 2:40am UTC](https://discuss.elastic.co/t/es-indexing-strategy/58413/3 "2016-08-19T02:40:00Z")

</div>

Well, each index uses resources and for a lot of our queries we have to go across multiple indexes which we've seen perform poorly for one user running such a query (across a few million documents). I do need to go back and review the docs and run some tests but was wondering what others are doing, recommend, and gain further insight.

Thanks.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 19, 2016, 3:50am UTC](https://discuss.elastic.co/t/es-indexing-strategy/58413/4 "2016-08-19T03:50:56Z")

</div>

Querying one index with 10 shards is the same as querying 10 indices with 1 shard. It also costs the same to maintain those shards, irrespective of the amount of data in those shards/indices (operational cost, not including application stuff like fielddata).

That said, you really want to keep things separate, it's better data hygiene and you can still query across multiple indices. If you use a pattern like `logstash-$type-$time` structure, you can still use `logstash-*` as the pattern in KB.

So, separate indices, reduce the shard count for them, and you should be good.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:26pm UTC](https://discuss.elastic.co/t/es-indexing-strategy/58413/5 "2017-07-05T22:26:48Z")

</div>


