Topbeat for JVM metrics monitoring


(Kushan85) #1

Can we use topbeat for monitoring JVM metrics like heap memory, active threads, garbage collection etc. ?


(Andrew Kroh) #2

No, it cannot. There is no Beat for JVM monitoring at the current time. This was discuss a bit here:


#3

You can give Httpbeat together with Jolokia a try. Httpbeat is a beat which can call any HTTP endpoint and Jolokia is a HTTP/JSON bridge for JMX. I'm currently using this combination to monitor a set of 30+ servers monitoring common metrics like connections, memory usage.

Disclaimer: I'm the owner of Httpbeat.


(Kushan85) #4

Thanks @christiangalsterer. That is interesting. I will give it a try. we have 100+ servers to monitor.


(Josh Star Long) #5

@christiangalsterer, I'm also facing a similar issue. Basically I want to monitor the metrics of a spark cluster. I have the metrics of the master on http://localhost:8081/metrics/master/json/path and those of the workers on http://localhost:4041/metrics/json/, http://localhost:4042/metrics/json/ http://localhost:4042/metrics/json/ and so on.

With this configure please can you help me with some steps or some documentation on how to get those metrics with httpbeat and jolokia.

Thanks.


#6

Hi can find the httpbeat here along with same examples how to query an HTTP endpoint.

E.g. you can point to http://localhost:8081/metrics/master/json/path to query the metrics of your spark master and send those data then to elastic search and/or logstash.

For the document_type I suggest that you use a value of "spark" or something similar.


(Josh Star Long) #7

Thanks for the reply,

Sorry for the delay, I had something more urgent to address. I have done that configuration but the data that I want come in the field body,

"body" : "{\"version\":\"3.0.0\",\"gauges\":{\"jvm.PS-MarkSweep.count\":{\"value\":1},\"jvm.PS-MarkSweep.time\":{\"value\":20},\"jvm.PS-Scavenge.count\":{\"value\":3},\"jvm.PS-Scavenge.time\":{\"value\":25},\"jvm.heap.committed\":{\"value\":342360064},\"jvm.heap.init\":{\"value\":526385152},\"jvm.heap.max\":{\"value\":954728448},\"jvm.heap.usage\":{\"value\":0.1372370628260571},\"jvm.heap.used\":{\"value\":131024128},\"jvm.non-heap.committed\":{\"value\":41353216},\"jvm.non-heap.init\":{\"value\":2555904},\"jvm.non-heap.max\":{\"value\":-1},\"jvm.non-heap.usage\":{\"value\":-4.0685104E7},\"jvm.non-heap.used\":{\"value\":40685104},\"jvm.pools.Code-Cache.committed\":{\"value\":7012352},\"jvm.pools.Code-Cache.init\":{\"value\":2555904},\"jvm.pools.Code-Cache.max\":{\"value\":251658240},\"jvm.pools.Code-Cache.usage\":{\"value\":0.02767003377278646},\"jvm.pools.Code-Cache.used\":{\"value\":6964096},\"jvm.pools.Compressed-Class-Space.committed\":{\"value\":4194304},\"jvm.pools.Compressed-Class-Space.init\":{\"value\":0},\"jvm.pools.Compressed-Class-Space.max\":{\"value\":1073741824},\"jvm.pools.Compressed-Class-Space.usage\":{\"value\":0.0038136616349220276},\"jvm.pools.Compressed-Class-Space.used\":{\"value\":4094888},\"jvm.pools.Metaspace.committed\":{\"value\":30146560},\"jvm.pools.Metaspace.init\":{\"value\":0},\"jvm.pools.Metaspace.max\":{\"value\":-1},\"jvm.pools.Metaspace.usage\":{\"value\":0.9827830439028533},\"jvm.pools.Metaspace.used\":{\"value\":29627528},\"jvm.pools.PS-Eden-Space.committed\":{\"value\":132120576},\"jvm.pools.PS-Eden-Space.init\":{\"value\":132120576},\"jvm.pools.PS-Eden-Space.max\":{\"value\":314572800},\"jvm.pools.PS-Eden-Space.usage\":{\"value\":0.263955561319987},\"jvm.pools.PS-Eden-Space.used\":{\"value\":83033240},\"jvm.pools.PS-Old-Gen.committed\":{\"value\":188743680},\"jvm.pools.PS-Old-Gen.init\":{\"value\":351272960},\"jvm.pools.PS-Old-Gen.max\":{\"value\":716177408},\"jvm.pools.PS-Old-Gen.usage\":{\"value\":0.037000530460743045},\"jvm.pools.PS-Old-Gen.used\":{\"value\":26498944},\"jvm.pools.PS-Survivor-Space.committed\":{\"value\":21495808},\"jvm.pools.PS-Survivor-Space.init\":{\"value\":21495808},\"jvm.pools.PS-Survivor-Space.max\":{\"value\":21495808},\"jvm.pools.PS-Survivor-Space.usage\":{\"value\":0.999820244021532},\"jvm.pools.PS-Survivor-Space.used\":{\"value\":21491944},\"jvm.total.committed\":{\"value\":383713280},\"jvm.total.init\":{\"value\":528941056},\"jvm.total.max\":{\"value\":954728447},\"jvm.total.used\":{\"value\":171714072},\"master.aliveWorkers\":{\"value\":1},\"master.apps\":{\"value\":1},\"master.waitingApps\":{\"value\":0},\"master.workers\":{\"value\":1}},\"counters\":{},\"histograms\":{\"CodeGenerator.compilationTime\":{\"count\":0,\"max\":0,\"mean\":0.0,\"min\":0,\"p50\":0.0,\"p75\":0.0,\"p95\":0.0,\"p98\":0.0,\"p99\":0.0,\"p999\":0.0,\"stddev\":0.0},\"CodeGenerator.generatedClassSize\":{\"count\":0,\"max\":0,\"mean\":0.0,\"min\":0,\"p50\":0.0,\"p75\":0.0,\"p95\":0.0,\"p98\":0.0,\"p99\":0.0,\"p999\":0.0,\"stddev\":0.0},\"CodeGenerator.generatedMethodSize\":{\"count\":0,\"max\":0,\"mean\":0.0,\"min\":0,\"p50\":0.0,\"p75\":0.0,\"p95\":0.0,\"p98\":0.0,\"p99\":0.0,\"p999\":0.0,\"stddev\":0.0},\"CodeGenerator.sourceCodeSize\":{\"count\":0,\"max\":0,\"mean\":0.0,\"min\":0,\"p50\":0.0,\"p75\":0.0,\"p95\":0.0,\"p98\":0.0,\"p99\":0.0,\"p999\":0.0,\"stddev\":0.0}},\"meters\":{},\"timers\":{}}",

I was wondering if it is possible to configure httpbeat so that those data come in different fields. That is

{ 
  "version":"3.0.0",
  "gauges":{"jvm.PS-MarkSweep.count":{"value":1},
  "jvm.PS-MarkSweep.time":{"value":20},
  "jvm.PS-Scavenge.count":{"value":3},
  "jvm.PS-Scavenge.time":{"value":25},
  "jvm.heap.committed":{"value":342360064},
  "jvm.heap.init":{"value":526385152},
  ....
}

Thanks.


#8

Currently it is not possible to change the root field. Instead of sending the data directly to Elasticsearch you can send the data first to Logstash and use one powerful features and plugins to manipulate the payload before forwarding it to Elasticsearch. You can also try to use the ingest node feature of Elasticsearch which provides similar capabilities
but I haven't use the ingest node myself yet.


(ruflin) #9

In case you have jolokia in place, you could also be interested in the jolokia module: https://github.com/elastic/beats/pull/3570


(Kernel Panic) #10

Hi Ruflin,
That module seems to be awesome, is it safe to integrate with ES 5? how can I do it? any repo? or should I clone a git repo? can you provide some steps?

Thanks
Regards


(ruflin) #11

The jolokia module is in the nightly builds (https://beats-nightlies.s3.amazonaws.com/index.html?prefix=metricbeat/) and will be part of 5.4 release. The data structure is compatible with ES 5.


(system) #12