Connecting metricbeat to microservices

[shubhamh]

Hello,

I am looking to connect Metricbeat with multiple microservices on a node. Each microservice is running jolokia and we are looking to pull all of that data into metricbeat. We currently have metricbeat setup to monitor the system level metrics on the node itself but we were looking to connect to the microservices as well. Would this be possible?

If this is not possible, would it make sense that we create another metricbeat container on the same node to which all microservices would send their jolokia data to and then forward that data onward to the metricbeat that is monitoring the system level metrics? We are looking to have all our data in a central location.

[jsoriano (Jaime Soriano)]

Hi @shubhamh and welcome

Yes, Metricbeat has a Jolokia module that can be used to collect metrics from JVM applications. For that you can use the metricbeat instance you already have in your node, there is no need to start a container for each microservice. Take into account that Jolokia address has to be reachable by metricbeat.

We recently published a blogpost about monitoring java applications with jolokia and metricbeat, you can read it here: https://www.elastic.co/blog/monitoring-java-applications-with-metricbeat-and-jolokia

How are you orchestrating your microservices? if you are using kubernetes and/or docker you can use the autodiscover providers to easily manage the configuration, specially after the introduction of hints-based configuration.

[shubhamh]

Hi Jaime,

Sorry for the late reply. We are currently using docker and using DCOS to manage the microservices. We are currently having trouble connecting the module to the microservices. We seem to be getting a connection refused. I have pasted our module configuration on our dcos node below. Can you please advise as to how we can connect our jolokia endpoints to metricbeat? Please let me know if you have any questions. Thank you

metricbeat.modules:

#--------------------System Module------------------

• module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
  hosts: ["X.X.X.X:9997"]
  namespace: "metrics"
  jmx.mappings:
• mbean: 'java.lang:type=Runtime'
  attributes:
• attr: Uptime
  field: uptime
• mbean: 'java.lang:type=GarbageCollector,name=ConcurrentMarkSweep'
  attributes:
• attr: CollectionTime
  field: gc.cms_collection_time
• attr: CollectionCount
  field: gc.cms_collection_count
• mbean: 'java.lang:type=Memory'
  attributes:
• attr: HeapMemoryUsage
  field: memory.heap_usage
• attr: NonHeapMemoryUsage
  field: memory.non_heap_usage

tags: ["jolokia-test"]

#-----------------Logstash Output------------------

metricbeat.autodiscover:
providers:

• type: docker
  templates:
• condition:
  contains:
  docker.container.image: nexus
  config:
• module: docker
  metricsets: ["leader", "self", "store"]
  hosts: "X.X.X.X:9997"

output.logstash:
hosts: ["samplewebsite.com:5044"]
ssl.enabled: true
ssl.certificate_authorities: ["/etc/ca.crt"]
ssl.certificate: "/etc/client.crt"
ssl.key: "/etc/client.key"

[jsoriano (Jaime Soriano)]

Hi @shubhamh,

Sorry but I cannot see your configuration very well, could you paste it as preformatted text using the </> button?

If the connection refusal happens connecting to jolokia, you may need to check if there is connectivity between metricbeat and jolokia. Jolokia by default uses to listen only on localhost, if you are running metricbeat in a different container you may need to configure jolokia to listen on an interface that metricbeat can reach, or on 0.0.0.0.

The autodiscover configuration you pasted looks incorrect, in the module template you need to put the module configuration you would use to collect metrics from the container matching the condition. I guess that in your case you'd like to put there the jolokia module configuration.