the sarama library creating an HTTP handler in /debug/metrics is kind of bad practice. Libraries should not silently register/open endpoints on behalf of the application. The endpoint was not merged into debug/vars, but removed (filtered out), also protecting beats from other libraries potentially installing their own global HTTP handler.
The way metrics are collected and reported by beats and sarama library did differ quite a lot. Beats prefer to report absolute values, no deltas. In ES/Kibana one can use the derivate of the absolute value to compute the rate.
Note: httpprof is no feature you should rely on (it even exposes the golang profiler). Currently it's more like some internal/private developers tools. We're in the process of adding standardised metrics capabilities to beats. This PR even adds an official /stats endpoint to the development branch for future use.
Which metrics is it you're exactly looking for? We're more interested to report similar/comparable metrics for all outputs in beats. As we don't have all of those metrics kafka can potentially report in the other outputs, we did expose only a very small/narrow subset for now.