V6.0.0-beta2 http module mapping trouble

[Sylvain_Mougenot]

I'm trying to collect metrics from Dorpwizard metrics.
Those are exposed by the REST API as Json.

I tryed dropwizard module for metrics but it seams not to fetch any datas.
Then switched to http module for metrics but not event is indexed due to mapping error

2017-09-07T11:33:09+02:00 WARN Can not index event (status=400): {"type":"illegal_argument_exception","reason":"mapper [http.metrics_http_me.classes] of different type, current_type [long], merged_type [ObjectMapper]"}

The index was dropped and service restarted without any improvement.

Can someone help to fix this, please.

Metricbeat version is 6.0.0-beta2 installed using rpm package on centos 6 VM.
config files are
/etc/metricbeat/metricbeat.yml

# Ansible managed

metricbeat.config.modules:
  path: /appli/conf/beats/metric*.yml
  reload.enabled: true
  reload.period: 30s

env: none
fields_under_root: false
name: localhost

processors: null

output.elasticsearch.hosts:
- localhost:9200

logging:
  files:
    keepfiles: 7
    name: metricbeat.log
    path: /var/log/metricbeat
    rotateeverybytes: 10485760
  level: info
  to_files: true

/appli/conf/beats/metricHttpMeasurement.yml

- module: http
  metricsets: ['json']
  hosts: ['localhost:8086/ep-manage/metrics']
  period: 10s
  username: 'admin'
  password: 'password'
  namespace: 'metrics_http_me'

[exekias (Carlos Pérez Aradros)]

Hi @Sylvain_Mougenot,

I think that your endpoint is returning different values for classes field, and switching between a number (long) and a map (object). That is probably causing issues with automatic mapping, does this make sense?

On the other hand, did you try to debug your dropwizard issue?

[Sylvain_Mougenot]

About dropwizard, even in debug, I've not logs for this module.

You are right, the endpoint (standard spring-boot actuator) seems to expose jmx metrics that have a confusing naming convention

  "classes": 15946,
  "classes.loaded": 16303,
  "classes.unloaded": 357,
  "heap": 254976,
  "heap.committed": 238080,
  "heap.init": 131072,
  "heap.used": 142330,
  "mem": 395507,
  "mem.free": 95749,
  "nonheap": 1556480,
  "nonheap.committed": 161792,
  "nonheap.init": 2496,
  "nonheap.used": 157427,
  "threads": 67,
  "threads.daemon": 46,
  "threads.peak": 70,
  "threads.totalStarted": 8182,

Is there a way to filter those in metrics settings?
I might be able to remove them from the spring-boot endpoint but it means to modify applications (I'd rather change one settings instead of redeploying all apps)

[exekias (Carlos Pérez Aradros)]

I think drop_fields processor should work for you, just add it with fields: ["classes"].

In any case it would be nice to debug your JMX issues if you are willing to, feel free to open a new thread for that.