Include_fields problem

fgouin · November 30, 2017, 5:37pm

Hi,

I'm having troubles with include_fields when the fields contains dots.

For instance, here is an extract of a Jolokia response:

"jolokia": {
"metrics": {
"Metrics": {
"atomikos.nbTransactions": 0.000000,
"classes": 18857.000000,
"classes.loaded": 19127.000000,
"classes.unloaded": 270.000000,
"counter.servo.discoveryclient-httpclient_createnew": 13542.000000,
"counter.servo.discoveryclient-httpclient_delete": 13534.000000,
"uptime": 406037194.000000
},
"java": {
"Threading": {
"DaemonThreadCount": 37.000000,
"ThreadCount": 57.000000
}
}
}
}

I can setup metricbeat to include these fields : 'jolokia.metrics.java.Threading.ThreadCount', 'jolokia.metrics.Metrics.classes', ...

But it won't include this field 'jolokia.metrics.Metrics.atomikos.nbTransactions', probably because of the dot in the attribute name.

Any idea ?

JKhondhu · November 30, 2017, 6:14pm

Hi,
Are you using the Metricbeat jolokia module? Can you share the config?

fgouin · December 1, 2017, 8:08am

Yes I am!

Here it is:

metricbeat.modules:
- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
  hosts: ["http://hostname:8383"]
  path: "/management/jolokia/?ignoreErrors=true&canonicalNaming=false"
  username: admin
  password: XXXX
  namespace: "metrics"
  jmx.mappings:
    - mbean: 'java.lang:type=Threading'
      attributes:
        - attr: ThreadCount
          field: java.Threading.ThreadCount
        - attr: DaemonThreadCount
          field: java.Threading.DaemonThreadCount
    - mbean: 'org.springframework.boot:name=metricsEndpoint,type=Endpoint'
      attributes:
        - attr: Data
          field: Metrics
processors:
- include_fields:
    fields: ['jolokia.metrics.java.Threading.ThreadCount', 'jolokia.metrics.Metrics.atomikos.nbTransactions', 'jolokia.metrics.Metrics.classes', 'metricset.module', 'metricset.name', 'metricset.namespace']

ruflin · December 4, 2017, 11:39pm

Any chance you could run metricbeat with debug logs and share it here? I wonder if quotes could also play a role here (but need to test on my end).

fgouin · December 12, 2017, 2:33pm

Sorry for the delay.

This is an extract of the debug log:

2017/12/12 14:27:54.898688 beat.go:285: INFO Home path: [/home/XXX/bin] Config path: [/home/XXX/bin] Data path: [/home/XXX/data] Logs path: [/home/XXX/bin/logs]
2017/12/12 14:27:54.898737 beat.go:186: INFO Setup Beat: metricbeat; Version: 5.5.2
2017/12/12 14:27:54.898874 metrics.go:23: INFO Metrics logging every 30s
2017/12/12 14:27:54.898871 processor.go:44: DBG  Processors: include_fields=Application, Env, error, UO, host, jolokia.metrics.java.Memory.HeapMemoryUsage.used, jolokia.metrics.java.Memory.NonHeapMemoryUsage.used, jolokia.metrics.java.OperatingSystem.OpenFileDescriptorCount, jolokia.metrics.java.OperatingSystem.ProcessCpuLoad, jolokia.metrics.java.Threading.DaemonThreadCount, jolokia.metrics.java.Threading.ThreadCount, jolokia.metrics.Metrics.atomikos.nbTransactions, jolokia.metrics.Metrics.atomikos.prrds.poolavailablesize, jolokia.metrics.Metrics.atomikos.prrds.pooltotalsize, metricset.host, metricset.module, metricset.name, metricset.namespace, src_type, type, @timestamp
2017/12/12 14:27:54.898962 beat.go:192: DBG  Initializing output plugins
2017/12/12 14:27:54.898985 publish.go:216: INFO Dry run mode. All output types except the file based one are disabled.
2017/12/12 14:27:54.899034 publish.go:295: INFO Publisher name: uljvsisi02
2017/12/12 14:27:54.899436 metricbeat.go:28: INFO Register [ModuleFactory:[docker, mongodb, mysql, postgresql, system], MetricSetFactory:[apache/status, ceph/cluster_disk, ceph/cluster_health, ceph/monitor_health, ceph/pool_disk, couchbase/bucket, couchbase/cluster, couchbase/node, docker/container, docker/cpu, docker/diskio, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, haproxy/info, haproxy/stat, jolokia/jmx, kafka/consumergroup, kafka/partition, mongodb/dbstats, mongodb/status, mysql/status, nginx/stubstatus, php_fpm/pool, postgresql/activity, postgresql/bgwriter, postgresql/database, prometheus/collector, prometheus/stats, redis/info, redis/keyspace, system/core, system/cpu, system/diskio, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, system/socket, zookeeper/mntr]]
2017/12/12 14:27:54.899728 jmx.go:49: WARN BETA: The jolokia jmx metricset is beta
2017/12/12 14:27:54.900144 builders.go:56: DBG  mb.NewModules() is returning map[{name:"jolokia", config:{Module:"jolokia", MetricSets:[jmx], Enabled:true, Hosts:[1 hosts], Period:"10s", Timeout:"10s", Raw:false, Fields:null, FieldsUnderRoot:false, Tags:[]}}:[{name:"jmx", module:"jolokia", hostData:{SanitizedURI:"http://hostname:8383/management/jolokia/%3FignoreErrors=true&canonicalNaming=false", Host:"hostname:8383"}}]]
2017/12/12 14:27:54.900199 wrapper.go:80: DBG  Initializing Module type 'jolokia': *mb.BaseModule={name:"jolokia", config:{Module:"jolokia", MetricSets:[jmx], Enabled:true, Hosts:[1 hosts], Period:"10s", Timeout:"10s", Raw:false, Fields:null, FieldsUnderRoot:false, Tags:[]}}
2017/12/12 14:27:54.900220 processor.go:44: DBG  Processors:
2017/12/12 14:27:54.900241 wrapper.go:96: DBG  Initializing MetricSet type 'jolokia/jmx' for host 'hostname:8383': *jmx.MetricSet={name:"jmx", module:"jolokia", hostData:{SanitizedURI:"http://hostname:8383/management/jolokia/%3FignoreErrors=true&canonicalNaming=false", Host:"hostname:8383"}}
2017/12/12 14:27:54.900464 beat.go:221: INFO metricbeat start running.
2017/12/12 14:27:54.900524 wrapper.go:128: DBG  Starting Wrapper[name=jolokia, len(metricSetWrappers)=1]
2017/12/12 14:27:54.900546 wrapper.go:149: DBG  Stopped Wrapper[name=jolokia, len(metricSetWrappers)=1]
2017/12/12 14:27:54.900898 wrapper.go:186: DBG  Starting metricSetWrapper[module=jolokia, name=jmx, host=hostname:8383]
2017/12/12 14:27:54.910700 client.go:214: DBG  Publish: {
  "@timestamp": "2017-12-12T14:27:54.900Z",
  "Application": "PRR",
  "Env": "in1",
  "UO": "3490",
  "jolokia": {
    "metrics": {
      "java": {
        "Memory": {
          "HeapMemoryUsage": {
            "used": 194627168.000000
          },
          "NonHeapMemoryUsage": {
            "used": 179718288.000000
          }
        },
        "OperatingSystem": {
          "OpenFileDescriptorCount": 102.000000,
          "ProcessCpuLoad": 0.000616
        },
        "Threading": {
          "DaemonThreadCount": 36.000000,
          "ThreadCount": 56.000000
        }
      }
    }
  },
  "metricset": {
    "host": "hostname:8383",
    "module": "jolokia",
    "name": "jmx",
    "namespace": "metrics"
  },
  "src_type": "metrics",
  "type": "metricsets"
}
2017/12/12 14:27:54.910843 async.go:35: DBG  publisher disabled
^C2017/12/12 14:27:59.083305 service.go:33: DBG  Received sigterm/sigint, stopping
2017/12/12 14:27:59.083494 wrapper.go:201: DBG  Stopped metricSetWrapper[module=jolokia, name=jmx, host=hostname:8383]
2017/12/12 14:27:59.083713 metrics.go:51: INFO Total non-zero values:  fetches.jolokia-jmx.events=1 fetches.jolokia-jmx.success=1 libbeat.publisher.published_events=1
2017/12/12 14:27:59.083733 metrics.go:52: INFO Uptime: 4.193512163s
2017/12/12 14:27:59.083740 beat.go:225: INFO metricbeat stopped.

ruflin · December 19, 2017, 2:37am

@fgouin It is indeed an issue with the dotted fields. See https://github.com/elastic/beats/pull/5916 Would be great to have your feedback on this PR on what you think would make most sense from your experience with Jolokia.

ruflin · December 26, 2017, 10:39pm

@fgouin Follow up issue on this is here: https://github.com/elastic/beats/issues/5942

system · January 23, 2018, 10:39pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.