Metricbeat Prometheus module not support match query

Metricbeat: 7.0.0


  • module: prometheus
    period: 10s
    hosts: ["<prometheus_url>"]
    metrics_path: '/federate'
    'match[]': '{name!=""}'
    namespace: example

This will result in:
"error": {
"message": "Unable to decode response from prometheus endpoint"

is this a bug?

config quotes from:

Dec 23rd, 2018: [EN][Observability] Querying metrics from Prometheus with Elastic Stack

The query works in a curl though.

Looks like this issue is due to the federate API returned too many events. is there a limit?

I further verified that when a query in a Prometheus directive returns over roughly 200k events, the error comes out. Had tuned the and output.elasticsearch bulk_max_size, both have no help on this case.

Thank you for your feedback @Jehutywong!

This sounds like a limit set somewhere in the client we use to scrape Prometheus metrics. Could you please open a new issue for it?

As a workaround, you could limit the number of metrics you get by updating the query to match only a subset.

Best regards

Thanks @exekias

I have a case opened in github #11912

Cut query into pieces works but hard to make sure all subsets not over limit.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.