Metricbeat|Failed to parse api response: invalid ch aracter '#' looking for beginning of value"

Elastic Stack Beats
1

Metricbeat:
"version": "7.10.0"

I am trying to use query metricbeat of 'prometheus' module using following config:

- module: prometheus
  period: 10s
  hosts: ["fluentd:24231"]
  metricsets: ["query"]
  queries:
  - name: "rate_output_buffer"
    path: "/metrics"
    params:
      query: "max_over_time(fluentd_output_status_buffer_total_bytes[1m])"  
In the metricbeat log I see following error:
 "error": {
    "message": "error parsing response from: http://fluentd:24231/metrics?query=max_over_time%28fluentd_output_status_buffer_total_bytes%5B1m%5D%29: Failed to parse api response: invalid character '#' looking for beginning of value"
  }

Following is a snippet of metrics available on above configured /metrics endpoint. What am I missing here ?

# TYPE fluentd_output_status_buffer_total_bytes gauge
# HELP fluentd_output_status_buffer_total_bytes Current total size of stage and queue buffers.
fluentd_output_status_buffer_total_bytes{hostname="eb2be64bd061",plugin_id="out_http-9883",type="file"} 187929.0
# TYPE fluentd_output_status_buffer_stage_length gauge
# HELP fluentd_output_status_buffer_stage_length Current length of stage buffers.
fluentd_output_status_buffer_stage_length{hostname="eb2be64bd061",plugin_id="out_http-9883",type="file"} 1.0
# TYPE fluentd_output_status_buffer_stage_byte_size gauge
# HELP fluentd_output_status_buffer_stage_byte_size Current total size of stage buffers.
fluentd_output_status_buffer_stage_byte_size{hostname="eb2be64bd061",plugin_id="out_http-9883",type="file"} 187929.0
# TYPE fluentd_output_status_buffer_queue_length gauge
# HELP fluentd_output_status_buffer_queue_length Current length of queue buffers.
fluentd_output_status_buffer_queue_length{hostname="eb2be64bd061",plugin_id="out_http-9883",type="file"} 0.0
# TYPE fluentd_output_status_queue_byte_size gauge
2

Hi!

Your path setting seems to be the issue. You have /metrics which is not the right one for the query API, it should be /api/v1/query or /api/v1/query_range.
Please check the https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-prometheus-query.html for more info.

3

Thanks Mark

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.