I'm using Kibana 4.2 and have an issue where I need to aggregate data regarding how many cpus were used by a framework. The job runs on multiple nodes, and the data is currently reported on a per-node basis.
The only way I've found to get the aggregate the total CPUs for each framework is to set the y-axis to average 'CPUs used', set the x-axis equal to the framework name, and split bars based on node as seen in the screenshot below.
This would work if I didn't need to compare 'used CPUs' to 'allocated CPU', but that info would be very valuable overlapped in the same graph (allocated will always be higher than used).
Is there any way to mask the node colors all as one? Or otherwise what's a good graph to use for this issue? I'd really like to overlay the actual vs used cpu for every framework.
Unfortunately the issue is that these metrics aren't reported only once per container, furthermore, they're not emitted on a set schedule (due to containers within hosts being created & destroyed as needed).
So the issue with summing is that SUM(msg.actualCPUs) reports back >1m cpus, which is far exaggerated beyond the actual case. That's why we use average per host and rolled up per framework has been the closest we can get thus far.
See the example below and please note the difference in scale
Hi Thomas,
I might not be explaining myself well. The used vs allocated is somewhat trivial and not really that important.
I'm trying to display the average across all nodes in a bucket as one value. Currently when I have to average across all nodes and sum those to get the aggregate, it's not intuitive and takes a lot of processing power to display (~6000 lines per bar in every bar in the graph).
Instead of this:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.