# Marvel index rate maybe high

**URL:** https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051
**Category:** Elasticsearch
**Tags:** elastic-stack-monitoring
**Created:** [January 13, 2016, 4:10am UTC](https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051 "2016-01-13T04:10:35Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![whybangbang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whybangbang/32/7116_2.png) [@whybangbang](https://discuss.elastic.co/u/whybangbang)
#### Post date: [January 13, 2016, 4:10am UTC](https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051/1 "2016-01-13T04:10:35Z")

</div>

we put data into es by logstash from kafka, kafka consume rate is about 300message/s and logstash metrics plugin calculate is about same with kafka ,but es index rate is 675.2/s  
 ![](https://us1.discourse-cdn.com/elastic/original/2X/1/1e67a114cda986b353e3fa4acc12a42f6cad108c.png)

 ![](https://us1.discourse-cdn.com/elastic/original/2X/9/9e7d489065a29adc0377c4aaab6cc67e8c7402e7.png)  
 ![](https://us1.discourse-cdn.com/elastic/original/2X/1/156d12d91b80f81a69da90865f39fa9d029b9220.png)

our index is 2 shard 1 replicas  
es 2.1 kibana 4.3+  
if can tell me how to get the index rate by es restful request,i try to use "index/\_stats" and find "index\_current",but the value is dramatically different

---

<div class="post-metadata">

### Author: ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)
#### Post date: [January 13, 2016, 9:28am UTC](https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051/2 "2016-01-13T09:28:04Z")

</div>

Hi,

Marvel takes replicas into account when calculating indexing rate. In your case, every write goes to 1 primary shard and in 1 replica so the 300 index/sec are in fact in total 600 index / sec in your cluster.

We are thinking of changing the way the indexing rate is calculated. The Index Stats (see [https://www.elastic.co/guide/en/elasticsearch/reference/2.1/indices-stats.html](https://www.elastic.co/guide/en/elasticsearch/reference/2.1/indices-stats.html)) can be used to know the number of index operations that have been executed on primary shards only (see primaries.indexing.index\_total).

---

<div class="post-metadata">

### Author: ![whybangbang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whybangbang/32/7116_2.png) [@whybangbang](https://discuss.elastic.co/u/whybangbang)
#### Post date: [January 13, 2016, 9:44am UTC](https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051/3 "2016-01-13T09:44:14Z")

</div>

thinks for your reply ,i have get the value by "curl localhost:9200/index/\_stats?pretty",now marvel index rate is 233.67/s,primaries.indexing.index\_total is 8865237, i don't know relationship between "index rate" and "primaries.indexing.index\_total",can you tell me ,thank you very much:grin:

---

<div class="post-metadata">

### Author: ![michalterbert](https://avatars.discourse-cdn.com/v4/letter/m/848f3c/32.png) [@michalterbert](https://discuss.elastic.co/u/michalterbert)
#### Post date: [January 13, 2016, 9:54pm UTC](https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051/4 "2016-01-13T21:54:19Z")

</div>

@whybangbang: could you tell me what application is used to show messages in/sec?  
This is additional kibana dashboard or somthing else?  
How do you do kafka benchmark in logstash? Using metric codec/filter?

---

<div class="post-metadata">

### Author: ![whybangbang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whybangbang/32/7116_2.png) [@whybangbang](https://discuss.elastic.co/u/whybangbang)
#### Post date: [January 14, 2016, 2:27am UTC](https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051/5 "2016-01-14T02:27:26Z")

</div>

"kafka manager" show message/s and "metric filter" show logstash message out/sec

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:45pm UTC](https://discuss.elastic.co/t/marvel-index-rate-maybe-high/39051/6 "2017-07-06T13:45:40Z")

</div>


