# Indices total indexing time metric given by marvel **URL:** \ **Category:** Elasticsearch\ **Created:** [April 16, 2014, 9:25am UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044 "2014-04-16T09:25:38Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![sindhu](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@sindhu](https://discuss.elastic.co/u/sindhu)\ **Post date:** [April 16, 2014, 9:25am UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044/1 "2014-04-16T09:25:38Z")
Hi, I'm new to elastic search and marvel . I'm currently investigating marvel and I want to know if the "Indices total indexing time" shown by marvel is "sum of the times taken by each shard" or "time between the arrival of request and completion of indexing". For example if there are 5 shards and indexing request arrives at 's' and indexing completes at 'e'. And shards take time t1,t2,t3,t4 and t5 respectively. So how is Indices total indexing time calculated now. Is ut e-s or t1+t2+t3+t4+t5. To add to this question, just want to know if marvel uses stats api. Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/97700530-a3e7-4d72-9a37-31ec85161c10%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/97700530-a3e7-4d72-9a37-31ec85161c10%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\ **Post date:** [April 16, 2014, 8:00pm UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044/2 "2014-04-16T20:00:38Z")
Hi, By the title, I assume you refer to the chart show in the Node Stats dashboard. This one is tricky. It shows you the total spent on indexing per second. This is best explained by an example - say you have 2 shards on a node. Say that each indexes 100% of the time (continuously indexing) and they do so in parallel. In this case we have 2 seconds of indexing (total of two shards) for each real-world second. In other words - see this as a load factor of indexing on the node. Technically we indeed use the node stats api which gives you the total spent on indexing of all shards that ever were on this node (reset on restart).In your example it will be t1+t2+t3+t4 (if all shards were on that node). The chart you see then plots the derivative (change per second) of this value. Hope this helps, Boaz On Wednesday, April 16, 2014 11:25:38 AM UTC+2, sindhu wrote: > Hi, > > I'm new to Elasticsearch and marvel . I'm currently investigating marvel > and I want to know if the "Indices total indexing time" shown by marvel is > "sum of the times taken by each shard" or "time between the arrival of > request and completion of indexing". > > For example if there are 5 shards and indexing request arrives at 's' and > indexing completes at 'e'. And shards take time t1,t2,t3,t4 and t5 > respectively. So how is Indices total indexing time calculated now. Is ut > e-s or t1+t2+t3+t4+t5. > > To add to this question, just want to know if marvel uses stats api. > > Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![sindhu](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@sindhu](https://discuss.elastic.co/u/sindhu)\ **Post date:** [April 17, 2014, 12:44pm UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044/3 "2014-04-17T12:44:58Z")
Hi, Thanks for the repy. Helped a lot. I also have a query about index rate total. Is it 1. no of documents indexed per sec 2. no of indexing req coming per sec 3. no of indexing requests completed per sec Can you also provide any links to documentation about what each metric exactly represents if possible. Thanks, sindhu. On Thu, Apr 17, 2014 at 1:30 AM, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote: > Hi, > > By the title, I assume you refer to the chart show in the Node Stats > dashboard. This one is tricky. It shows you the total spent on indexing per > second. This is best explained by an example - say you have 2 shards on a > node. Say that each indexes 100% of the time (continuously indexing) and > they do so in parallel. In this case we have 2 seconds of indexing (total > of two shards) for each real-world second. In other words - see this as a > load factor of indexing on the node. > > Technically we indeed use the node stats api which gives you the total > spent on indexing of all shards that ever were on this node (reset on > restart).In your example it will be t1+t2+t3+t4 (if all shards were on that > node). The chart you see then plots the derivative (change per second) of > this value. > > Hope this helps, > Boaz > > On Wednesday, April 16, 2014 11:25:38 AM UTC+2, sindhu wrote: > > > Hi, > > > > I'm new to Elasticsearch and marvel . I'm currently investigating marvel > > and I want to know if the "Indices total indexing time" shown by marvel is > > "sum of the times taken by each shard" or "time between the arrival of > > request and completion of indexing". > > > > For example if there are 5 shards and indexing request arrives at 's' and > > indexing completes at 'e'. And shards take time t1,t2,t3,t4 and t5 > > respectively. So how is Indices total indexing time calculated now. Is ut > > e-s or t1+t2+t3+t4+t5. > > > > To add to this question, just want to know if marvel uses stats api. > > > > Thanks. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/1pKOwD7y\_Bg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/1pKOwD7y_Bg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMWeVUvXXKUBdTcNz2W1gGaBhhmYgGjjW2DkaMgJUQrV1C6U-A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMWeVUvXXKUBdTcNz2W1gGaBhhmYgGjjW2DkaMgJUQrV1C6U-A%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![sindhu](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@sindhu](https://discuss.elastic.co/u/sindhu)\ **Post date:** [April 17, 2014, 1:50pm UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044/4 "2014-04-17T13:50:43Z")
Hi, Thanks for the reply. Helped a lot. I also have a query about index rate total. Is it 1. no of documents indexed per sec 2. no of indexing requests coming per sec 3. no of indexing requests completed per sec In my case i gave 42 files for indexing { "\_shards": { "total": 10, "successful": 10, "failed": 0 }, "\_all": { "primaries": { "docs": { "count": 48460, "deleted": 0 }, "store": { "size\_in\_bytes": 1089957, "throttle\_time\_in\_millis": 0 }, "indexing": { "index\_total": 42, "index\_time\_in\_millis": 1645, "index\_current": 0, "delete\_total": 0, "delete\_time\_in\_millis": 0, "delete\_current": 0 }, So what exactly does the graph of metric "Indexing request Rate" represnts. Can you also provide any links to documentation about what each metric exactly represents if possible. Thanks, sindhu. On Thu, Apr 17, 2014 at 6:14 PM, sindhuri nadella \< [sindhuri.nadella@gmail.com](mailto:sindhuri.nadella@gmail.com)\> wrote: > Hi, > > Thanks for the repy. Helped a lot. I also have a query about index rate > total. Is it > > 1. no of documents indexed per sec > 2. no of indexing req coming per sec > 3. no of indexing requests completed per sec > > Can you also provide any links to documentation about what each metric > exactly represents if possible. > > Thanks, > sindhu. > > On Thu, Apr 17, 2014 at 1:30 AM, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote: > > > Hi, > > > > By the title, I assume you refer to the chart show in the Node Stats > > dashboard. This one is tricky. It shows you the total spent on indexing per > > second. This is best explained by an example - say you have 2 shards on a > > node. Say that each indexes 100% of the time (continuously indexing) and > > they do so in parallel. In this case we have 2 seconds of indexing (total > > of two shards) for each real-world second. In other words - see this as a > > load factor of indexing on the node. > > > > Technically we indeed use the node stats api which gives you the total > > spent on indexing of all shards that ever were on this node (reset on > > restart).In your example it will be t1+t2+t3+t4 (if all shards were on that > > node). The chart you see then plots the derivative (change per second) of > > this value. > > > > Hope this helps, > > Boaz > > > > On Wednesday, April 16, 2014 11:25:38 AM UTC+2, sindhu wrote: > > > > > Hi, > > > > > > I'm new to Elasticsearch and marvel . I'm currently investigating > > > marvel and I want to know if the "Indices total indexing time" shown by > > > marvel is "sum of the times taken by each shard" or "time between the > > > arrival of request and completion of indexing". > > > > > > For example if there are 5 shards and indexing request arrives at 's' > > > and indexing completes at 'e'. And shards take time t1,t2,t3,t4 and t5 > > > respectively. So how is Indices total indexing time calculated now. Is ut > > > e-s or t1+t2+t3+t4+t5. > > > > > > To add to this question, just want to know if marvel uses stats api. > > > > > > Thanks. > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/1pKOwD7y\_Bg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/1pKOwD7y_Bg/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8c04d37e-1b69-44de-af7c-ab94ac474cfc%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMWeVUt3G5Tqq4ryT%2Bgc7vJmxVDUAt9GiFzkBn%2B-i4ncnF5xGA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMWeVUt3G5Tqq4ryT%2Bgc7vJmxVDUAt9GiFzkBn%2B-i4ncnF5xGA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![sindhu](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@sindhu](https://discuss.elastic.co/u/sindhu)\ **Post date:** [April 17, 2014, 2:07pm UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044/5 "2014-04-17T14:07:30Z")
Hi, Thanks for the reply. Helped a lot. I also have a query about index rate total. Is it 1. no of documents indexed per sec 2. no of indexing requests coming per sec 3. no of indexing requests completed per sec In my case i gave 42 files for indexing and following is the output of codeindex\_stats where codeindex is the index name. { "\_shards": { "total": 10, "successful": 10, "failed": 0 }, "\_all": { "primaries": { "docs": { "count": 48460, "deleted": 0 }, "store": { "size\_in\_bytes": 1089957, "throttle\_time\_in\_millis": 0 }, "indexing": { "index\_total": 42, "index\_time\_in\_millis": 1645, "index\_current": 0, "delete\_total": 0, "delete\_time\_in\_millis": 0, "delete\_current": 0 }, So what exactly does the graph of metric "Indexing request Rate" represents. How do you calculate the index rate? Can you also provide any links to documentation about what each metric exactly represents if possible. Thanks, sindhu. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c20d5d6e-0cc9-4213-ad21-ee6830b07259%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c20d5d6e-0cc9-4213-ad21-ee6830b07259%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\ **Post date:** [April 19, 2014, 3:45pm UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044/6 "2014-04-19T15:45:25Z")
The Indexing request Rate chart shows the number of complete indexing doc operations (index/create) per second. It counts on the document level so a bulk request with 10 docs is equal to 10 requests with a single doc. The counter is updated when the operation is completed. I think the chart title is misleading as it refers to requests and that's not accurate. I will change it. On Thursday, April 17, 2014 4:07:30 PM UTC+2, sindhu wrote: > Hi, > > Thanks for the reply. Helped a lot. I also have a query about index rate > total. Is it > > 1. no of documents indexed per sec > 2. no of indexing requests coming per sec > 3. no of indexing requests completed per sec > > In my case i gave 42 files for indexing and following is the output of > codeindex\_stats where codeindex is the index name. > > { > "\_shards": { > "total": 10, > "successful": 10, > "failed": 0 > }, > "\_all": { > "primaries": { > "docs": { > "count": 48460, > "deleted": 0 > }, > "store": { > "size\_in\_bytes": 1089957, > "throttle\_time\_in\_millis": 0 > }, > "indexing": { > "index\_total": 42, > "index\_time\_in\_millis": 1645, > "index\_current": 0, > "delete\_total": 0, > "delete\_time\_in\_millis": 0, > "delete\_current": 0 > }, > > So what exactly does the graph of metric "Indexing request Rate" > represents. How do you calculate the index rate? > > Can you also provide any links to documentation about what each metric > exactly represents if possible. > > Thanks, > sindhu. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9cc610b1-1b19-4138-a4eb-00d585bc0906%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9cc610b1-1b19-4138-a4eb-00d585bc0906%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:34am UTC](https://discuss.elastic.co/t/indices-total-indexing-time-metric-given-by-marvel/17044/7 "2017-07-06T01:34:50Z")