What components to track for performance monitoring?


(Otis Gospodnetić) #1

Hi,

In our SPM for Solr service we have a graph that shows performance of
different Solr search components (see attachment for a stacked version of
this graph).

I know ES doesn't have the equivalent of SearchComponents, although it has
request actions, but is there anything in ES that one could track
performance of similarly to what's in the attached image for Solr?

We can try finding key components/classes/methods in ES for performance
monitoring purposes, but I thought I'd ask here first.

Thanks,
Otis

Search Analytics - http://sematext.com/search-analytics/index.html
Scalable Performance Monitoring - http://sematext.com/spm/index.html


(George Stathis) #2

The only thing I'm aware of at the moment is the amazing bigdesk plugin
(https://github.com/lukas-vlcek/bigdesk) but unfortunately, it doesn't
track internal components like your image shows. It's more along the lines
of local OS and JVM stats, GC, reads and writes. Still, I find it super
useful to see in real-time some of the activity that's happening on the
nodes.

-GS

On Monday, July 30, 2012 7:04:19 PM UTC-4, Otis Gospodnetic wrote:

Hi,

In our SPM for Solr service we have a graph that shows performance of
different Solr search components (see attachment for a stacked version of
this graph).

I know ES doesn't have the equivalent of SearchComponents, although it has
request actions, but is there anything in ES that one could track
performance of similarly to what's in the attached image for Solr?

We can try finding key components/classes/methods in ES for performance
monitoring purposes, but I thought I'd ask here first.

Thanks,
Otis

Search Analytics - http://sematext.com/search-analytics/index.html
Scalable Performance Monitoring - http://sematext.com/spm/index.html


(Otis Gospodnetić) #3

Hi,

I probably didn't do a good job of asking the question.
I know of ElasticSearch performance monitoring solutions like BigDesk as
well as our SPM for ES (see
http://sematext.com/spm/elasticsearch-performance-monitoring/index.html ).

What I was wondering was what components inside ES can be monitored, for
example to see what the slowest parts in search request processing are.
For example, if you look at the screenshot I sent, which is a graph from
SPM for Solr, you can see "QueryComponent" takes the most time, followed by
SpellCheckComponent.

I was wondering if there are any components in ES whose performance one
could track in a similar fashion?

Thanks,
Otis

On Monday, July 30, 2012 7:59:54 PM UTC-4, George Stathis wrote:

The only thing I'm aware of at the moment is the amazing bigdesk plugin (
https://github.com/lukas-vlcek/bigdesk) but unfortunately, it doesn't
track internal components like your image shows. It's more along the lines
of local OS and JVM stats, GC, reads and writes. Still, I find it super
useful to see in real-time some of the activity that's happening on the
nodes.

-GS

On Monday, July 30, 2012 7:04:19 PM UTC-4, Otis Gospodnetic wrote:

Hi,

In our SPM for Solr service we have a graph that shows performance of
different Solr search components (see attachment for a stacked version of
this graph).

I know ES doesn't have the equivalent of SearchComponents, although it
has request actions, but is there anything in ES that one could track
performance of similarly to what's in the attached image for Solr?

We can try finding key components/classes/methods in ES for performance
monitoring purposes, but I thought I'd ask here first.

Thanks,
Otis

Search Analytics - http://sematext.com/search-analytics/index.html
Scalable Performance Monitoring - http://sematext.com/spm/index.html


(hazzadous) #4

Old thread, sure but what's happened with this?

Things I'd like to see:

  • per shard search
  • per shard facetting times
  • reduce times for the above
  • wait times on merges(?)

I haven't looked at 1.0 so not sure if already addressed but would be
loverly to have a more granular breakdown. The main issue we have is
tracking timings on individual queries. Maybe there is already some
setting to allow this?

On Tuesday, 31 July 2012 11:41:19 UTC+1, Otis Gospodnetic wrote:

Hi,

I probably didn't do a good job of asking the question.
I know of ElasticSearch performance monitoring solutions like BigDesk as
well as our SPM for ES (see
http://sematext.com/spm/elasticsearch-performance-monitoring/index.html ).

What I was wondering was what components inside ES can be monitored, for
example to see what the slowest parts in search request processing are.
For example, if you look at the screenshot I sent, which is a graph from
SPM for Solr, you can see "QueryComponent" takes the most time, followed by
SpellCheckComponent.

I was wondering if there are any components in ES whose performance one
could track in a similar fashion?

Thanks,
Otis

On Monday, July 30, 2012 7:59:54 PM UTC-4, George Stathis wrote:

The only thing I'm aware of at the moment is the amazing bigdesk plugin (
https://github.com/lukas-vlcek/bigdesk) but unfortunately, it doesn't
track internal components like your image shows. It's more along the lines
of local OS and JVM stats, GC, reads and writes. Still, I find it super
useful to see in real-time some of the activity that's happening on the
nodes.

-GS

On Monday, July 30, 2012 7:04:19 PM UTC-4, Otis Gospodnetic wrote:

Hi,

In our SPM for Solr service we have a graph that shows performance of
different Solr search components (see attachment for a stacked version of
this graph).

I know ES doesn't have the equivalent of SearchComponents, although it
has request actions, but is there anything in ES that one could track
performance of similarly to what's in the attached image for Solr?

We can try finding key components/classes/methods in ES for performance
monitoring purposes, but I thought I'd ask here first.

Thanks,
Otis

Search Analytics - http://sematext.com/search-analytics/index.html
Scalable Performance Monitoring - http://sematext.com/spm/index.html

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/477dfaa9-6f90-488d-9249-b0b583e343b1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Otis Gospodnetić) #5

Hi,

I think the per-shard stats are still not available, unless it got added
recently and I missed it?
Have a look at SPM for Elasticsearch, it may be detailed enough for you.
SPM doesn't show timing for individual queries, but our free Search
Analytics service does give you per-query timing info, per-query
click-through rates, per-query zero-hits, and so on - http://sematext.com/search-analytics/

  • there is a live demo (look for the big blue button) you can play with.

Otis

Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/

On Friday, January 24, 2014 10:22:54 PM UTC-5, hazzadous wrote:

Old thread, sure but what's happened with this?

Things I'd like to see:

  • per shard search
  • per shard facetting times
  • reduce times for the above
  • wait times on merges(?)

I haven't looked at 1.0 so not sure if already addressed but would be
loverly to have a more granular breakdown. The main issue we have is
tracking timings on individual queries. Maybe there is already some
setting to allow this?

On Tuesday, 31 July 2012 11:41:19 UTC+1, Otis Gospodnetic wrote:

Hi,

I probably didn't do a good job of asking the question.
I know of ElasticSearch performance monitoring solutions like BigDesk as
well as our SPM for ES (see
http://sematext.com/spm/elasticsearch-performance-monitoring/index.html
).

What I was wondering was what components inside ES can be monitored, for
example to see what the slowest parts in search request processing are.
For example, if you look at the screenshot I sent, which is a graph from
SPM for Solr, you can see "QueryComponent" takes the most time, followed by
SpellCheckComponent.

I was wondering if there are any components in ES whose performance one
could track in a similar fashion?

Thanks,
Otis

On Monday, July 30, 2012 7:59:54 PM UTC-4, George Stathis wrote:

The only thing I'm aware of at the moment is the amazing bigdesk plugin (
https://github.com/lukas-vlcek/bigdesk) but unfortunately, it doesn't
track internal components like your image shows. It's more along the lines
of local OS and JVM stats, GC, reads and writes. Still, I find it super
useful to see in real-time some of the activity that's happening on the
nodes.

-GS

On Monday, July 30, 2012 7:04:19 PM UTC-4, Otis Gospodnetic wrote:

Hi,

In our SPM for Solr service we have a graph that shows performance of
different Solr search components (see attachment for a stacked version of
this graph).

I know ES doesn't have the equivalent of SearchComponents, although it
has request actions, but is there anything in ES that one could track
performance of similarly to what's in the attached image for Solr?

We can try finding key components/classes/methods in ES for performance
monitoring purposes, but I thought I'd ask here first.

Thanks,
Otis

Search Analytics - http://sematext.com/search-analytics/index.html
Scalable Performance Monitoring - http://sematext.com/spm/index.html

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/085d33d4-1512-4c5b-a018-214b1a2fada1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6