MetricBeat Jolokia configuration

vishnuduttpv · March 23, 2018, 5:48am

Hi,
The following is my conf.
ElasticSearch - 6.2.2
Logstash - 6.2.2
Kibana - 6.2.2
Metricbeat - 6.2.2

I am trying to pull the JMX data from my clients using the jolokia module in the metricbeat. I had installed the jolokia in my target clients and enabled the jolokia module in the metricbeat. While manually checking through the browser Iam getting the result throgh jolokia as expected. Following is my metricbeat jolokia conf,
- module: jolokia
_ metricsets: ["jmx"]_
_ period: 10s_
_ hosts: ["10.10.114.155:8085"]_
_ namespace: "metrics"_
_ path: "/jolokia/read/"_
_ jmx.mapping:_
_ - mbean: 'java.lang:type=Runtime'_
_ attributes:_
_ - attr: Uptime_
_ field: uptime_

But while restarting the metricbeat I am getting the following error,
2018-03-23T11:04:36.866+0530 ERROR instance/beat.go:667 Exiting: 1 error: 1 error: missing required field accessing '0.jmx.mappings' (source:'/etc/metricbeat/modules.d/jolokia.yml')

Request to advise !
Thanks in advance!

Vishnu!

agathver · March 23, 2018, 5:57am

@vishnuduttpv

You perhaps have a small typo in the config. You need to define jmx.mappings, in order to properly unmarshall JSON response from Jolokia.

Here is a config I use:

- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
  namespace: "jolokia_metrics"
  hosts: ["localhost:8081"]
  jmx.mappings:
    - mbean: 'java.lang:type=Runtime'
      attributes:
        - attr: Uptime
          field: uptime
    - mbean: 'java.lang:type=GarbageCollector,name=ConcurrentMarkSweep'
      attributes:
        - attr: CollectionTime
          field: gc.cms_collection_time
        - attr: CollectionCount
          field: gc.cms_collection_count
    - mbean: 'java.lang:type=Memory'
      attributes:
        - attr: HeapMemoryUsage
          field: memory.heap_usage
        - attr: NonHeapMemoryUsage
          field: memory.non_heap_usage

vishnuduttpv · March 23, 2018, 6:41am

@agathver Thanks for the reply with the sample! I tried with the sample. Its working. But in the metric beat logs I am getting the following error that the fields are missing.
2018-03-23T12:11:18.550+0530 INFO template/load.go:73 Template already exists and will not be overwritten.
2018-03-23T12:11:27.564+0530 ERROR schema/schema.go:41 Error on field 'server_uptime': Missing field: server_uptime, Error: Key ServerUptimeSeconds not found
2018-03-23T12:11:27.564+0530 ERROR schema/schema.go:41 Error on field 'user': Missing field: user, Error: Key CPUUser not found
2018-03-23T12:11:27.564+0530 ERROR schema/schema.go:41 Error on field 'system': Missing field: system, Error: Key CPUSystem not found
2018-03-23T12:11:27.565+0530 ERROR schema/schema.go:41 Error on field 'children_user': Missing field: children_user, Error: Key CPUChildrenUser not found
2018-03-23T12:11:27.565+0530 ERROR schema/schema.go:41 Error on field 'children_system': Missing field: children_system, Error: Key CPUChildrenSystem not found
2018-03-23T12:11:37.565+0530 ERROR schema/schema.go:41 Error on field 'children_system': Missing field: children_system, Error: Key CPUChildrenSystem not found
2018-03-23T12:11:37.565+0530 ERROR schema/schema.go:41 Error on field 'user': Missing field: user, Error: Key CPUUser not found
2018-03-23T12:11:37.565+0530 ERROR schema/schema.go:41 Error on field 'system': Missing field: system, Error: Key CPUSystem not found
2018-03-23T12:11:37.565+0530 ERROR schema/schema.go:41 Error on field 'children_user': Missing field: children_user, Error: Key CPUChildrenUser not found
2018-03-23T12:11:37.565+0530 ERROR schema/schema.go:41 Error on field 'server_uptime': Missing field: server_uptime, Error: Key ServerUptimeSeconds not found

Is there any tomcat version dependencies ? I am using apache tomcat 7! and installed the jolokia as a webarchive accessible through http://x.x.x.x:8085/jolokia
Thanks in advance!
Vishnu

agathver · March 23, 2018, 7:49am

This is some other error related to the system module in Metricbeats. Check the modules.d/system.yaml

I don't think so. This does not seem related to the Jolokia module.

vishnuduttpv · March 23, 2018, 9:16am

@agathver Thanks for the reply! But the systems.yml is working fine and its sending the correct metrics too. The jolokia conf is as follows. I added some more detailed metrics in the conf.
- module: jolokia
_ metricsets: ["jmx"]_
_ enabled: true_
_ period: 10s_
_ namespace: "jolokia_metrics"_
_ hosts: ["10.10.114.155:8085"]_
_ path: "/jolokia/?ignoreErrors=true&canonicalNaming=false"_
_ jmx.mappings:_
_ #jmx.mapping:_

_ - mbean: 'java.lang:type=Runtime'_
_ attributes:_
_ - attr: Uptime_
_ field: uptime_
_ field_type: integer_

_ - mbean: 'java.lang:type=GarbageCollector,name=PS MarkSweep'_
_ attributes:_
_ - attr: CollectionTime_
_ field: gc.cms_collection_time_
_ - attr: CollectionCount_
_ field: gc.cms_collection_count_
_ field_type: integer_

_ - mbean: 'java.lang:type=Memory'_
_ attributes:_
_ - attr: HeapMemoryUsagePercent_
_ field: memory.heap_usage_pct_
_ field_type: integer_
_ - attr: NonHeapMemoryUsage_
_ field: memory.non_heap_usage_
_ field_type: integer_

_ - mbean: 'java.lang:type=OperatingSystem'_
_ attributes:_
_ - attr: ProcessCpuLoad_
_ field: cpu.processload.pct_
_ field_type: integer_

_ - mbean: 'java.lang:type=ClassLoading'_
_ attributes:_
_ - attr: LoadedClassCount_
_ field: classloaded.classcount_
_ field_type: integer_
_ - attr: TotalLoadedClassCCount_
_ field: classloading.totalclasscount_
_ field_type: integer_

_ - mbean: 'java.lang:type=Threading'_
_ attributes:_
_ - attr: DaemonThreadCount _
field: threadcount.daemon
_ field_type: integer_
_ - attr: DeadlockedThreadCount_
_ field: threadcount.deadlocked_
_ field_type: integer_
- attr: PeakThreadCount
_ field: threadcount.peak_
_ field_type: integer_
_ - attr: ThreadCount_
_ field: threadcount.threadcount_
_ field_type: integer_

But I think its related with the JMX data,because i tried by disabling the jolokia module.
Really I am stuck!

Thanks in advance!

agathver · March 23, 2018, 3:13pm

Could you please check the Apache module? On closer inspection, the values seem like those from Apache status page.

Please use three backticks to enclose any code/conf snippets that you post it makes it easier to read , as it invokes syntax highlighting and preserves whitespace.

Like:

```yaml
- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
 ...
 ...
 ...

```

Which outputs:

- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s

vishnuduttpv · March 26, 2018, 9:20am

@agathver Sorry for the misalignment of the codes. Thanks for the reply!
I tried by disabling the Apache! As suspected it was coming from the apache module itself! But in kibana, I am getting the results from the apache server. And also related with the jolokia, Iam not getting any logs of the jolokia module in kibana!
Whether there is any default dashboards for the Jolokia Module, in kibana? If yes how to import it in the kibana?
Thanks in advance!
Vishnu

agathver · March 27, 2018, 4:26pm

As of now there are no default dashboards for the Jolokia module.

We are working for some standardization of JMX MBeans by the JVM and other applications like Tomcat.

Hopefully I'll be working on creating dashboards for Jolokia in the coming days.

vishnuduttpv · March 28, 2018, 5:39am

@agathver Thanks for the reply! I was working with the jolokia module for monitoring our test app servers. So its successful with the apache and other modules. Is there any way to create a dashboard manually from my end so that I can forward with the testing for jolokia! Sorry ,as I am newbie to the KIBANA. Request you to advise!
Thanks in advance
Vishnu

agathver · March 28, 2018, 4:32pm

There are some dashboard building instructions here: https://www.elastic.co/guide/en/kibana/current/dashboard-getting-started.html

The key is to identify the ES queries you need and representing the result with the appropriate visualization.

system · April 25, 2018, 4:33pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.