# Definitive guide for tuning using Marvel?

**URL:** <https://discuss.elastic.co/t/definitive-guide-for-tuning-using-marvel/52346>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-monitoring\
**Created:** [June 9, 2016, 3:01pm UTC](https://discuss.elastic.co/t/definitive-guide-for-tuning-using-marvel/52346 "2016-06-09T15:01:19Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![paulkeogh](https://avatars.discourse-cdn.com/v4/letter/p/8dc957/32.png) [@paulkeogh](https://discuss.elastic.co/u/paulkeogh)\
**Post date:** [June 9, 2016, 3:01pm UTC](https://discuss.elastic.co/t/definitive-guide-for-tuning-using-marvel/52346/1 "2016-06-09T15:01:19Z")

</div>

Is there a definitive guide available for tuning elasticsearch using Marvel ?

I have a 6 node cluster with 5 billion docs running over 2 VMs on VMWares vCloud environment.

A simple aggregation is taking ~40 seconds - how can I use Marvel to investigate this latency in a formal and methodical fashion ?

Thanks,

---

<div class="post-metadata">

**Author:** ![pickypg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pickypg/32/62409_2.png) [@pickypg](https://discuss.elastic.co/u/pickypg)\
**Post date:** [June 16, 2016, 7:25pm UTC](https://discuss.elastic.co/t/definitive-guide-for-tuning-using-marvel/52346/2 "2016-06-16T19:25:08Z")

</div>

There is no guide -- at least not yet -- but options vary based on the version of Elasticsearch that you're running.

For example, ES 2.3 (not Marvel) supports [a new query profiler](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-profile.html). This could be immensely useful, but it does not yet profile aggregation performance. Even so, it will help to see the performance of the raw query that you're performing, which is a prerequisite to the overall aggregation.

In terms of using Marvel to help to tune ES, it's more of an operational tool for tuning _overall_ performance, rather than individual requests. Having said that, you can use a fresh instance that _only_ runs the individual request as a way to tune based on that request.

Things that you need to be aware of for tuning _any_ request really has nothing to do with Marvel, but everything to do with some general guidelines:

- The `size` parameter of every level of the request has a direct impact on the amount of data that needs to be passed around (both externally, which is the obvious part, but also internally within the cluster).
  - This is a big problem that I see users frequently run into by requesting scary `size`s.

- The amount of work done by the `query` is a baseline requirement for getting the overall request to run.
  - If you are not using filters when possible, then you should rework the request to use them because filters can be cached for repeated requests.

- Any sorting that you may be doing.
  - It's unclear which version you're using, but [it's possible that you're using fielddata rather than doc values](https://www.elastic.co/blog/support-in-the-wild-my-biggest-elasticsearch-problem-at-scale), which may be causing a lot of performance issues (it's more likely that you're using fielddata in ES 1.x).

- The amount of work done by the `aggs` (aka `aggregations`) at each step.
  - It's unclear which version you're using, but [it's possible that you're using fielddata rather than doc values](https://www.elastic.co/blog/support-in-the-wild-my-biggest-elasticsearch-problem-at-scale), which may be causing a lot of performance issues (it's more likely that you're using fielddata in ES 1.x).

Which of those would Marvel help with? Really only the last two because of the repeated sub-bullet. But it also shows search latency, which should highlight when this is becoming a problem.

Another point that I noticed from your post was that you mentioned 6 nodes were running on 2 VMs. Why are there 3 nodes per VM? You could look at Marvel to see what each node is doing in terms of its performance metrics while this type of slow request is being handled: does their JVM Heap utilization go up significantly? Does the CPU go crazy?

Hope that helps,  
Chris

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:43pm UTC](https://discuss.elastic.co/t/definitive-guide-for-tuning-using-marvel/52346/3 "2017-07-06T13:43:07Z")

</div>


