BigDesk question


(Greg Mowery) #1

I just started using this tool (to the authors and contributors, thanks
very much!)

I have what probably are very simplistic questions.

I am doing some early load/sizing testing, and using big desk to monitor my
cluster, and a pretty simple app to throw queries at the cluster.

On my simple app side, I SOP the response time, and its about 100 ms.

I am trying to align this data with what I see in BigDesk. There are two
graphs that I do not fully understand what data is being presented.

Search Requests per Second (Delta). The delta concerns me. The graph is
hovering around 1500, does this directly relate to searches per second..if
so, what is the (delta) for.

Same question for search time per second (delta). This hovers around 10. I
have no concept of what this means in relation to my (supposedly) 100 ms
response time.

Thanks!

Greg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Lukáš Vlček) #2

Hi,

Bigdesk is build on top of data pulled from REST API (mainly cluster API
[1]). And it pulls the data in user specified interval (can be changed
within UI or provided directly as an URL parameter, for example
'refresh=5000' for 5 sec refresh).

The charts that you ask about are based on data taken from nodes stats API
[2]:
:/_nodes//stats
where the is selected node in the web UI.

The response from the above REST endpoint contains stats data about indices
(like search, indexing and get operations). And some of the attributes are
in form of cumulative numbers (like total from the node JVM start). Meaning
you get the same or greater value with each consecutive request/response.
So in order to plot some nice charts we calculate a delta value. In other
words the delta value means the difference between the most recent and
previous response value. In addition to this we divide the result by the
value of refresh interval (in sec). So what you see in the chart can be
read something like "the average change per sec".

Here is an example for "Search requests per second (Δ)":

  • You do some "_search" request
  • It hits 15 shards of some indices on that node, so the value of indices
    -> search -> "query_total" in nodes stats API [2] response increases by 15
  • Bigdesk refresh value is 5000 (5 sec)

As a result the chart should display peak of 3 (15/5) in the Query line. So
if the value is ~1500 in your case then it means in average an X number of
shards is hit by search requests per second where X=1500*refresh (does it
make sense)?

You can see the chart is really only informative (it depends on refresh
interval and number of shards). But there is the cumulative "query_total"
value displayed as well in the web UI.

Similarly, the second chart "Search time per second (Δ)" displays the
average time (in mills) spent in query or fetch phase on the node. Again
this value includes all involved shards on that node.

HTH

Regards
Lukas

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster.html
[2]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html

On Tue, Nov 5, 2013 at 8:43 PM, Greg Mowery greg.mowery@gmail.com wrote:

I just started using this tool (to the authors and contributors, thanks
very much!)

I have what probably are very simplistic questions.

I am doing some early load/sizing testing, and using big desk to monitor
my cluster, and a pretty simple app to throw queries at the cluster.

On my simple app side, I SOP the response time, and its about 100 ms.

I am trying to align this data with what I see in BigDesk. There are two
graphs that I do not fully understand what data is being presented.

Search Requests per Second (Delta). The delta concerns me. The graph is
hovering around 1500, does this directly relate to searches per second..if
so, what is the (delta) for.

Same question for search time per second (delta). This hovers around 10. I
have no concept of what this means in relation to my (supposedly) 100 ms
response time.

Thanks!

Greg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Greg Mowery) #3

Great explanation!!

Thanks for taking the time Lukas, I appreciate it.

Greg

On Thu, Nov 7, 2013 at 5:50 AM, Lukáš Vlček lukas.vlcek@gmail.com wrote:

Hi,

Bigdesk is build on top of data pulled from REST API (mainly cluster API
[1]). And it pulls the data in user specified interval (can be changed
within UI or provided directly as an URL parameter, for example
'refresh=5000' for 5 sec refresh).

The charts that you ask about are based on data taken from nodes stats API
[2]:
:/_nodes//stats
where the is selected node in the web UI.

The response from the above REST endpoint contains stats data about
indices (like search, indexing and get operations). And some of the
attributes are in form of cumulative numbers (like total from the node JVM
start). Meaning you get the same or greater value with each consecutive
request/response. So in order to plot some nice charts we calculate a
delta value. In other words the delta value means the difference between
the most recent and previous response value. In addition to this we divide
the result by the value of refresh interval (in sec). So what you see in
the chart can be read something like "the average change per sec".

Here is an example for "Search requests per second (Δ)":

  • You do some "_search" request
  • It hits 15 shards of some indices on that node, so the value of indices
    -> search -> "query_total" in nodes stats API [2] response increases by 15
  • Bigdesk refresh value is 5000 (5 sec)

As a result the chart should display peak of 3 (15/5) in the Query line.
So if the value is ~1500 in your case then it means in average an X number
of shards is hit by search requests per second where X=1500*refresh (does
it make sense)?

You can see the chart is really only informative (it depends on refresh
interval and number of shards). But there is the cumulative "query_total"
value displayed as well in the web UI.

Similarly, the second chart "Search time per second (Δ)" displays the
average time (in mills) spent in query or fetch phase on the node. Again
this value includes all involved shards on that node.

HTH

Regards
Lukas

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster.html
[2]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html

On Tue, Nov 5, 2013 at 8:43 PM, Greg Mowery greg.mowery@gmail.com wrote:

I just started using this tool (to the authors and contributors, thanks
very much!)

I have what probably are very simplistic questions.

I am doing some early load/sizing testing, and using big desk to monitor
my cluster, and a pretty simple app to throw queries at the cluster.

On my simple app side, I SOP the response time, and its about 100 ms.

I am trying to align this data with what I see in BigDesk. There are two
graphs that I do not fully understand what data is being presented.

Search Requests per Second (Delta). The delta concerns me. The graph is
hovering around 1500, does this directly relate to searches per second..if
so, what is the (delta) for.

Same question for search time per second (delta). This hovers around 10.
I have no concept of what this means in relation to my (supposedly) 100 ms
response time.

Thanks!

Greg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/p_4JA_x-K14/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Greg M.


.ǝɟı1 sıɥ ɟo ʇsǝɹ ǝɥʇ ɹoɟ ɯɹɐʍ ǝq 11,ǝɥ puɐ 'ǝɹıɟ uo ɯıɥ ʇǝs ʇnq 'ǝʇnuıɯ ɐ
ɹoɟ ɯɹɐʍ ǝq 11,ǝɥ puɐ 'ɥɔʇɐɯ ɐ uɐɯ ɐ ǝʌıb

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #4