Include_fields problem

(Fabrice Gouin) #1


I'm having troubles with include_fields when the fields contains dots.

For instance, here is an extract of a Jolokia response:

"jolokia": {
"metrics": {
"Metrics": {
"atomikos.nbTransactions": 0.000000,
"classes": 18857.000000,
"classes.loaded": 19127.000000,
"classes.unloaded": 270.000000,
"counter.servo.discoveryclient-httpclient_createnew": 13542.000000,
"counter.servo.discoveryclient-httpclient_delete": 13534.000000,
"uptime": 406037194.000000
"java": {
"Threading": {
"DaemonThreadCount": 37.000000,
"ThreadCount": 57.000000

I can setup metricbeat to include these fields : '', 'jolokia.metrics.Metrics.classes', ...

But it won't include this field 'jolokia.metrics.Metrics.atomikos.nbTransactions', probably because of the dot in the attribute name.

Any idea ?

(Jymit Singh Khondhu) #2

Are you using the Metricbeat jolokia module? Can you share the config?

(Fabrice Gouin) #3

Yes I am!

Here it is:

- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
  hosts: ["http://hostname:8383"]
  path: "/management/jolokia/?ignoreErrors=true&canonicalNaming=false"
  username: admin
  password: XXXX
  namespace: "metrics"
    - mbean: 'java.lang:type=Threading'
        - attr: ThreadCount
          field: java.Threading.ThreadCount
        - attr: DaemonThreadCount
          field: java.Threading.DaemonThreadCount
    - mbean: 'org.springframework.boot:name=metricsEndpoint,type=Endpoint'
        - attr: Data
          field: Metrics
- include_fields:
    fields: ['', 'jolokia.metrics.Metrics.atomikos.nbTransactions', 'jolokia.metrics.Metrics.classes', 'metricset.module', '', 'metricset.namespace']

(ruflin) #4

Any chance you could run metricbeat with debug logs and share it here? I wonder if quotes could also play a role here (but need to test on my end).

(Fabrice Gouin) #5

Sorry for the delay.

This is an extract of the debug log:

2017/12/12 14:27:54.898688 beat.go:285: INFO Home path: [/home/XXX/bin] Config path: [/home/XXX/bin] Data path: [/home/XXX/data] Logs path: [/home/XXX/bin/logs]
2017/12/12 14:27:54.898737 beat.go:186: INFO Setup Beat: metricbeat; Version: 5.5.2
2017/12/12 14:27:54.898874 metrics.go:23: INFO Metrics logging every 30s
2017/12/12 14:27:54.898871 processor.go:44: DBG  Processors: include_fields=Application, Env, error, UO, host,,,,,,, jolokia.metrics.Metrics.atomikos.nbTransactions, jolokia.metrics.Metrics.atomikos.prrds.poolavailablesize, jolokia.metrics.Metrics.atomikos.prrds.pooltotalsize,, metricset.module,, metricset.namespace, src_type, type, @timestamp
2017/12/12 14:27:54.898962 beat.go:192: DBG  Initializing output plugins
2017/12/12 14:27:54.898985 publish.go:216: INFO Dry run mode. All output types except the file based one are disabled.
2017/12/12 14:27:54.899034 publish.go:295: INFO Publisher name: uljvsisi02
2017/12/12 14:27:54.899436 metricbeat.go:28: INFO Register [ModuleFactory:[docker, mongodb, mysql, postgresql, system], MetricSetFactory:[apache/status, ceph/cluster_disk, ceph/cluster_health, ceph/monitor_health, ceph/pool_disk, couchbase/bucket, couchbase/cluster, couchbase/node, docker/container, docker/cpu, docker/diskio, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, haproxy/info, haproxy/stat, jolokia/jmx, kafka/consumergroup, kafka/partition, mongodb/dbstats, mongodb/status, mysql/status, nginx/stubstatus, php_fpm/pool, postgresql/activity, postgresql/bgwriter, postgresql/database, prometheus/collector, prometheus/stats, redis/info, redis/keyspace, system/core, system/cpu, system/diskio, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, system/socket, zookeeper/mntr]]
2017/12/12 14:27:54.899728 jmx.go:49: WARN BETA: The jolokia jmx metricset is beta
2017/12/12 14:27:54.900144 builders.go:56: DBG  mb.NewModules() is returning map[{name:"jolokia", config:{Module:"jolokia", MetricSets:[jmx], Enabled:true, Hosts:[1 hosts], Period:"10s", Timeout:"10s", Raw:false, Fields:null, FieldsUnderRoot:false, Tags:[]}}:[{name:"jmx", module:"jolokia", hostData:{SanitizedURI:"http://hostname:8383/management/jolokia/%3FignoreErrors=true&canonicalNaming=false", Host:"hostname:8383"}}]]
2017/12/12 14:27:54.900199 wrapper.go:80: DBG  Initializing Module type 'jolokia': *mb.BaseModule={name:"jolokia", config:{Module:"jolokia", MetricSets:[jmx], Enabled:true, Hosts:[1 hosts], Period:"10s", Timeout:"10s", Raw:false, Fields:null, FieldsUnderRoot:false, Tags:[]}}
2017/12/12 14:27:54.900220 processor.go:44: DBG  Processors:
2017/12/12 14:27:54.900241 wrapper.go:96: DBG  Initializing MetricSet type 'jolokia/jmx' for host 'hostname:8383': *jmx.MetricSet={name:"jmx", module:"jolokia", hostData:{SanitizedURI:"http://hostname:8383/management/jolokia/%3FignoreErrors=true&canonicalNaming=false", Host:"hostname:8383"}}
2017/12/12 14:27:54.900464 beat.go:221: INFO metricbeat start running.
2017/12/12 14:27:54.900524 wrapper.go:128: DBG  Starting Wrapper[name=jolokia, len(metricSetWrappers)=1]
2017/12/12 14:27:54.900546 wrapper.go:149: DBG  Stopped Wrapper[name=jolokia, len(metricSetWrappers)=1]
2017/12/12 14:27:54.900898 wrapper.go:186: DBG  Starting metricSetWrapper[module=jolokia, name=jmx, host=hostname:8383]
2017/12/12 14:27:54.910700 client.go:214: DBG  Publish: {
  "@timestamp": "2017-12-12T14:27:54.900Z",
  "Application": "PRR",
  "Env": "in1",
  "UO": "3490",
  "jolokia": {
    "metrics": {
      "java": {
        "Memory": {
          "HeapMemoryUsage": {
            "used": 194627168.000000
          "NonHeapMemoryUsage": {
            "used": 179718288.000000
        "OperatingSystem": {
          "OpenFileDescriptorCount": 102.000000,
          "ProcessCpuLoad": 0.000616
        "Threading": {
          "DaemonThreadCount": 36.000000,
          "ThreadCount": 56.000000
  "metricset": {
    "host": "hostname:8383",
    "module": "jolokia",
    "name": "jmx",
    "namespace": "metrics"
  "src_type": "metrics",
  "type": "metricsets"
2017/12/12 14:27:54.910843 async.go:35: DBG  publisher disabled
^C2017/12/12 14:27:59.083305 service.go:33: DBG  Received sigterm/sigint, stopping
2017/12/12 14:27:59.083494 wrapper.go:201: DBG  Stopped metricSetWrapper[module=jolokia, name=jmx, host=hostname:8383]
2017/12/12 14:27:59.083713 metrics.go:51: INFO Total non-zero values: fetches.jolokia-jmx.success=1 libbeat.publisher.published_events=1
2017/12/12 14:27:59.083733 metrics.go:52: INFO Uptime: 4.193512163s
2017/12/12 14:27:59.083740 beat.go:225: INFO metricbeat stopped.

(ruflin) #6

@fgouin It is indeed an issue with the dotted fields. See Would be great to have your feedback on this PR on what you think would make most sense from your experience with Jolokia.

(ruflin) #7

@fgouin Follow up issue on this is here:

(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.