# High cpu usage with marvel-agent (2.4) **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-monitoring\ **Created:** [April 3, 2018, 2:03pm UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587 "2018-04-03T14:03:18Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![fnork](https://avatars.discourse-cdn.com/v4/letter/f/4da419/32.png) [@fnork](https://discuss.elastic.co/u/fnork)\ **Post date:** [April 3, 2018, 2:03pm UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/1 "2018-04-03T14:03:18Z")
We have an 8 node cluster with about 100 indexes and 1000 shards. The node with the primary marvel index shard shows a running \>80% cpu usage (in marvel :), while indexing about 60 docs/sec. hot threads shows the marvel-exporter thread using about 30%. I suppose the other cpu usage is indexing? Is this level of cpu usage normal for marvel or is there something in our cluster that causes excessive cpu use? -andré ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 4, 2018, 10:40am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/2 "2018-04-04T10:40:43Z")
can you share the hot threads output in a gist? That said, you may also want to reduce your number of shards, and especially you may want to think about upgrading, if possible. As there have been a lot of changes in the two major releases in between and APIs that can help you like the shrink index API. ---
**Author:** ![fnork](https://avatars.discourse-cdn.com/v4/letter/f/4da419/32.png) [@fnork](https://discuss.elastic.co/u/fnork)\ **Post date:** [April 4, 2018, 10:52am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/3 "2018-04-04T10:52:01Z")
Thanks for your reply. Here's the hot threads output for the node. > I realize there are a lot of shards and that we let marvel index back into the production cluster, but the cpu load still seems a bit high to me. Can the free Marvel export to a separate cluster or is a full license needed for that? We need the Marvel logging for debugging right now, but we could possible increase the metric collection interval when we are done with that. -andré ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 4, 2018, 11:06am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/4 "2018-04-04T11:06:23Z")
Is it possible that you have a big cluster state? ---
**Author:** ![fnork](https://avatars.discourse-cdn.com/v4/letter/f/4da419/32.png) [@fnork](https://discuss.elastic.co/u/fnork)\ **Post date:** [April 4, 2018, 11:07am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/5 "2018-04-04T11:07:44Z")
Yes we have enormous mappings for some of the indexes. There is an unfortunate dynamic data scheme in this application. \_mapping returns 30MB of json. By a rough grep one index has about 40k fields and there are probably 10-20 with 5k to 15k fields. -andré ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 4, 2018, 11:48am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/6 "2018-04-04T11:48:10Z")
then just serializing this cluster state seems to need a lot of CPU according to hot\_threads - which makes sense to me. ---
**Author:** ![fnork](https://avatars.discourse-cdn.com/v4/letter/f/4da419/32.png) [@fnork](https://discuss.elastic.co/u/fnork)\ **Post date:** [April 4, 2018, 12:19pm UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/7 "2018-04-04T12:19:25Z")
ok is this independent of marvel-agent or is it something marvel-agent does? -andré ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 4, 2018, 3:12pm UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/8 "2018-04-04T15:12:31Z")
The marvel agent reads the cluster state and serializes parts of it (using filters) - I havent looked up how the filters are working, but more efficient filtering using jackson has been added much later than 2.4. --Alex ---
**Author:** ![fnork](https://avatars.discourse-cdn.com/v4/letter/f/4da419/32.png) [@fnork](https://discuss.elastic.co/u/fnork)\ **Post date:** [April 5, 2018, 11:01am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/9 "2018-04-05T11:01:20Z")
ok thanks for taking the time to look into this 🙂 -andré ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 3, 2018, 11:01am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-marvel-agent-2-4/126587/10 "2018-05-03T11:01:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.