# Include\_fields problem

**URL:** <https://discuss.elastic.co/t/include-fields-problem/109809>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [November 30, 2017, 5:37pm UTC](https://discuss.elastic.co/t/include-fields-problem/109809 "2017-11-30T17:37:30Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![fgouin](https://avatars.discourse-cdn.com/v4/letter/f/bcef8e/32.png) [@fgouin](https://discuss.elastic.co/u/fgouin)\
**Post date:** [November 30, 2017, 5:37pm UTC](https://discuss.elastic.co/t/include-fields-problem/109809/1 "2017-11-30T17:37:30Z")

</div>

Hi,

I'm having troubles with include\_fields when the fields contains dots.

For instance, here is an extract of a Jolokia response:

"jolokia": {  
"metrics": {  
"Metrics": {  
"atomikos.nbTransactions": 0.000000,  
"classes": 18857.000000,  
"classes.loaded": 19127.000000,  
"classes.unloaded": 270.000000,  
"counter.servo.discoveryclient-httpclient\_createnew": 13542.000000,  
"counter.servo.discoveryclient-httpclient\_delete": 13534.000000,  
"uptime": 406037194.000000  
},  
"java": {  
"Threading": {  
"DaemonThreadCount": 37.000000,  
"ThreadCount": 57.000000  
}  
}  
}  
}

I can setup metricbeat to include these fields : 'jolokia.metrics.java.Threading.ThreadCount', 'jolokia.metrics.Metrics.classes', ...

But it won't include this field 'jolokia.metrics.Metrics.atomikos.nbTransactions', probably because of the dot in the attribute name.

Any idea ?

---

<div class="post-metadata">

**Author:** ![JKhondhu](https://avatars.discourse-cdn.com/v4/letter/j/ed655f/32.png) [@JKhondhu](https://discuss.elastic.co/u/JKhondhu)\
**Post date:** [November 30, 2017, 6:14pm UTC](https://discuss.elastic.co/t/include-fields-problem/109809/2 "2017-11-30T18:14:44Z")

</div>

Hi,  
Are you using the Metricbeat jolokia module? Can you share the config?

---

<div class="post-metadata">

**Author:** ![fgouin](https://avatars.discourse-cdn.com/v4/letter/f/bcef8e/32.png) [@fgouin](https://discuss.elastic.co/u/fgouin)\
**Post date:** [December 1, 2017, 8:08am UTC](https://discuss.elastic.co/t/include-fields-problem/109809/3 "2017-12-01T08:08:02Z")

</div>

Yes I am!

Here it is:

```
metricbeat.modules:
- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
  hosts: ["http://hostname:8383"]
  path: "/management/jolokia/?ignoreErrors=true&canonicalNaming=false"
  username: admin
  password: XXXX
  namespace: "metrics"
  jmx.mappings:
    - mbean: 'java.lang:type=Threading'
      attributes:
        - attr: ThreadCount
          field: java.Threading.ThreadCount
        - attr: DaemonThreadCount
          field: java.Threading.DaemonThreadCount
    - mbean: 'org.springframework.boot:name=metricsEndpoint,type=Endpoint'
      attributes:
        - attr: Data
          field: Metrics
processors:
- include_fields:
    fields: ['jolokia.metrics.java.Threading.ThreadCount', 'jolokia.metrics.Metrics.atomikos.nbTransactions', 'jolokia.metrics.Metrics.classes', 'metricset.module', 'metricset.name', 'metricset.namespace']
```

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 4, 2017, 11:39pm UTC](https://discuss.elastic.co/t/include-fields-problem/109809/4 "2017-12-04T23:39:28Z")

</div>

Any chance you could run metricbeat with debug logs and share it here? I wonder if quotes could also play a role here (but need to test on my end).

---

<div class="post-metadata">

**Author:** ![fgouin](https://avatars.discourse-cdn.com/v4/letter/f/bcef8e/32.png) [@fgouin](https://discuss.elastic.co/u/fgouin)\
**Post date:** [December 12, 2017, 2:33pm UTC](https://discuss.elastic.co/t/include-fields-problem/109809/5 "2017-12-12T14:33:05Z")

</div>

Sorry for the delay.

This is an extract of the debug log:

```
2017/12/12 14:27:54.898688 beat.go:285: INFO Home path: [/home/XXX/bin] Config path: [/home/XXX/bin] Data path: [/home/XXX/data] Logs path: [/home/XXX/bin/logs]
2017/12/12 14:27:54.898737 beat.go:186: INFO Setup Beat: metricbeat; Version: 5.5.2
2017/12/12 14:27:54.898874 metrics.go:23: INFO Metrics logging every 30s
2017/12/12 14:27:54.898871 processor.go:44: DBG Processors: include_fields=Application, Env, error, UO, host, jolokia.metrics.java.Memory.HeapMemoryUsage.used, jolokia.metrics.java.Memory.NonHeapMemoryUsage.used, jolokia.metrics.java.OperatingSystem.OpenFileDescriptorCount, jolokia.metrics.java.OperatingSystem.ProcessCpuLoad, jolokia.metrics.java.Threading.DaemonThreadCount, jolokia.metrics.java.Threading.ThreadCount, jolokia.metrics.Metrics.atomikos.nbTransactions, jolokia.metrics.Metrics.atomikos.prrds.poolavailablesize, jolokia.metrics.Metrics.atomikos.prrds.pooltotalsize, metricset.host, metricset.module, metricset.name, metricset.namespace, src_type, type, @timestamp
2017/12/12 14:27:54.898962 beat.go:192: DBG Initializing output plugins
2017/12/12 14:27:54.898985 publish.go:216: INFO Dry run mode. All output types except the file based one are disabled.
2017/12/12 14:27:54.899034 publish.go:295: INFO Publisher name: uljvsisi02
2017/12/12 14:27:54.899436 metricbeat.go:28: INFO Register [ModuleFactory:[docker, mongodb, mysql, postgresql, system], MetricSetFactory:[apache/status, ceph/cluster_disk, ceph/cluster_health, ceph/monitor_health, ceph/pool_disk, couchbase/bucket, couchbase/cluster, couchbase/node, docker/container, docker/cpu, docker/diskio, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, haproxy/info, haproxy/stat, jolokia/jmx, kafka/consumergroup, kafka/partition, mongodb/dbstats, mongodb/status, mysql/status, nginx/stubstatus, php_fpm/pool, postgresql/activity, postgresql/bgwriter, postgresql/database, prometheus/collector, prometheus/stats, redis/info, redis/keyspace, system/core, system/cpu, system/diskio, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, system/socket, zookeeper/mntr]]
2017/12/12 14:27:54.899728 jmx.go:49: WARN BETA: The jolokia jmx metricset is beta
2017/12/12 14:27:54.900144 builders.go:56: DBG mb.NewModules() is returning map[{name:"jolokia", config:{Module:"jolokia", MetricSets:[jmx], Enabled:true, Hosts:[1 hosts], Period:"10s", Timeout:"10s", Raw:false, Fields:null, FieldsUnderRoot:false, Tags:[]}}:[{name:"jmx", module:"jolokia", hostData:{SanitizedURI:"http://hostname:8383/management/jolokia/%3FignoreErrors=true&canonicalNaming=false", Host:"hostname:8383"}}]]
2017/12/12 14:27:54.900199 wrapper.go:80: DBG Initializing Module type 'jolokia': *mb.BaseModule={name:"jolokia", config:{Module:"jolokia", MetricSets:[jmx], Enabled:true, Hosts:[1 hosts], Period:"10s", Timeout:"10s", Raw:false, Fields:null, FieldsUnderRoot:false, Tags:[]}}
2017/12/12 14:27:54.900220 processor.go:44: DBG Processors:
2017/12/12 14:27:54.900241 wrapper.go:96: DBG Initializing MetricSet type 'jolokia/jmx' for host 'hostname:8383': *jmx.MetricSet={name:"jmx", module:"jolokia", hostData:{SanitizedURI:"http://hostname:8383/management/jolokia/%3FignoreErrors=true&canonicalNaming=false", Host:"hostname:8383"}}
2017/12/12 14:27:54.900464 beat.go:221: INFO metricbeat start running.
2017/12/12 14:27:54.900524 wrapper.go:128: DBG Starting Wrapper[name=jolokia, len(metricSetWrappers)=1]
2017/12/12 14:27:54.900546 wrapper.go:149: DBG Stopped Wrapper[name=jolokia, len(metricSetWrappers)=1]
2017/12/12 14:27:54.900898 wrapper.go:186: DBG Starting metricSetWrapper[module=jolokia, name=jmx, host=hostname:8383]
2017/12/12 14:27:54.910700 client.go:214: DBG Publish: {
  "@timestamp": "2017-12-12T14:27:54.900Z",
  "Application": "PRR",
  "Env": "in1",
  "UO": "3490",
  "jolokia": {
    "metrics": {
      "java": {
        "Memory": {
          "HeapMemoryUsage": {
            "used": 194627168.000000
          },
          "NonHeapMemoryUsage": {
            "used": 179718288.000000
          }
        },
        "OperatingSystem": {
          "OpenFileDescriptorCount": 102.000000,
          "ProcessCpuLoad": 0.000616
        },
        "Threading": {
          "DaemonThreadCount": 36.000000,
          "ThreadCount": 56.000000
        }
      }
    }
  },
  "metricset": {
    "host": "hostname:8383",
    "module": "jolokia",
    "name": "jmx",
    "namespace": "metrics"
  },
  "src_type": "metrics",
  "type": "metricsets"
}
2017/12/12 14:27:54.910843 async.go:35: DBG publisher disabled
^C2017/12/12 14:27:59.083305 service.go:33: DBG Received sigterm/sigint, stopping
2017/12/12 14:27:59.083494 wrapper.go:201: DBG Stopped metricSetWrapper[module=jolokia, name=jmx, host=hostname:8383]
2017/12/12 14:27:59.083713 metrics.go:51: INFO Total non-zero values: fetches.jolokia-jmx.events=1 fetches.jolokia-jmx.success=1 libbeat.publisher.published_events=1
2017/12/12 14:27:59.083733 metrics.go:52: INFO Uptime: 4.193512163s
2017/12/12 14:27:59.083740 beat.go:225: INFO metricbeat stopped.
```

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 19, 2017, 2:37am UTC](https://discuss.elastic.co/t/include-fields-problem/109809/6 "2017-12-19T02:37:40Z")

</div>

@fgouin It is indeed an issue with the dotted fields. See [https://github.com/elastic/beats/pull/5916](https://github.com/elastic/beats/pull/5916) Would be great to have your feedback on this PR on what you think would make most sense from your experience with Jolokia.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 26, 2017, 10:39pm UTC](https://discuss.elastic.co/t/include-fields-problem/109809/7 "2017-12-26T22:39:42Z")

</div>

@fgouin Follow up issue on this is here: [https://github.com/elastic/beats/issues/5942](https://github.com/elastic/beats/issues/5942)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 23, 2018, 10:39pm UTC](https://discuss.elastic.co/t/include-fields-problem/109809/8 "2018-01-23T22:39:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.