Aggregation profiling?


(Mike Sukmanowsky) #1

I don't believe there are any current endpoints in the API that support
this, but are there plans to add better profiling information to ES
aggregation queries? We'll see some agg queries return in 11s, then <5s
then >11s again. Sometimes we can see associated filter cache expirations,
but it's really hard to line these up to one specific query in our
production environment since multiple users are executing queries
simultaneously.

It'd be really helpful to optionally see where aggregation queries are
spending the bulk of their time to help us understand what to improve in
the future.

Anything we can do here right now?

--
Mike Sukmanowsky
Aspiring Digital Carpenter

e: mike.sukmanowsky@gmail.com

facebook http://facebook.com/mike.sukmanowsky | twitter
http://twitter.com/msukmanowsky | LinkedIn
http://www.linkedin.com/profile/view?id=10897143 | github
https://github.com/msukmanowsky

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOH6cu5WSGqQ%2BZ0_qrofXEvwo8JuSH9xoSbZgSwiT90MJ_wxdA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(James Macdonald) #2

I don't have an answer, but I really like this question. I too would love
to see more query and aggregation profiling tools for performance
optimization purposes.

Also, I assume you have already looked at this, but have you made sure you
are not evicting anything from your in memory field data?

James

On Mon, May 25, 2015 at 4:08 PM, Mike Sukmanowsky <
mike.sukmanowsky@gmail.com> wrote:

I don't believe there are any current endpoints in the API that support
this, but are there plans to add better profiling information to ES
aggregation queries? We'll see some agg queries return in 11s, then <5s
then >11s again. Sometimes we can see associated filter cache expirations,
but it's really hard to line these up to one specific query in our
production environment since multiple users are executing queries
simultaneously.

It'd be really helpful to optionally see where aggregation queries are
spending the bulk of their time to help us understand what to improve in
the future.

Anything we can do here right now?

--
Mike Sukmanowsky
Aspiring Digital Carpenter

e: mike.sukmanowsky@gmail.com

facebook http://facebook.com/mike.sukmanowsky | twitter
http://twitter.com/msukmanowsky | LinkedIn
http://www.linkedin.com/profile/view?id=10897143 | github
https://github.com/msukmanowsky

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAOH6cu5WSGqQ%2BZ0_qrofXEvwo8JuSH9xoSbZgSwiT90MJ_wxdA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAOH6cu5WSGqQ%2BZ0_qrofXEvwo8JuSH9xoSbZgSwiT90MJ_wxdA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAABsnTZOmx-fk%2BG9dR6-XYB_1j7mGRNRwTqvQRwKx0YAcopFWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3