# Metricbeat prometheus debug logging

**URL:** <https://discuss.elastic.co/t/metricbeat-prometheus-debug-logging/220387>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [February 21, 2020, 2:06pm UTC](https://discuss.elastic.co/t/metricbeat-prometheus-debug-logging/220387 "2020-02-21T14:06:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![chrmrn](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@chrmrn](https://discuss.elastic.co/u/chrmrn)\
**Post date:** [February 21, 2020, 2:06pm UTC](https://discuss.elastic.co/t/metricbeat-prometheus-debug-logging/220387/1 "2020-02-21T14:06:07Z")

</div>

I've gotten an autodiscover metricbeat for prometheus working (scraping a pod in k8s), but I keep getting the error:

`unable to decode response from prometheus endpoint: decoding of metric family failed: text format parsing error in line 1: expected float as value, got "'t"`

And I can't understand where it's getting the value 't from as it's nowhere in the /metrics response.

I can run Prometheus promtool against the /metrics path to verify that the prometheus format is valid.

I would like a way to gain insight into what metricbeat is trying to parse, but even setting maximum debug logging I'm not getting any info about which metric it's trying to parse...making it really hard to understand the problem. Is there a way to get verbose logging as to what values it's trying to parse? (or actually see the entire response it's trying to parse?). Doing a curl towards the /metrics endpoint from within the metricbeat pod I'm getting a valid response...so I need to understand why metricbeats can't parse the response (or which response it's actually trying to parse).

---

<div class="post-metadata">

**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\
**Post date:** [February 24, 2020, 9:44am UTC](https://discuss.elastic.co/t/metricbeat-prometheus-debug-logging/220387/2 "2020-02-24T09:44:13Z")

</div>

Hi!

I'm afraid this has to do with the text parsing and many things can go wrong (see similar [https://github.com/prometheus/prometheus/issues/1469](https://github.com/prometheus/prometheus/issues/1469))

In order to investigate the problem through Metricbeat you should manually debug it from sourcecode. [Here is](https://github.com/elastic/beats/blob/edc17b3da33c134c8085e4025e572b9b2d557366/metricbeat/helper/prometheus/prometheus.go#L95) where it fails, so you can most probably log the families that are successfully parsed so far and make out what is the one that cause the failure.

Regards,  
C.

---

<div class="post-metadata">

**Author:** ![chrmrn](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@chrmrn](https://discuss.elastic.co/u/chrmrn)\
**Post date:** [February 24, 2020, 10:14am UTC](https://discuss.elastic.co/t/metricbeat-prometheus-debug-logging/220387/3 "2020-02-24T10:14:43Z")

</div>

Thank you for the information!

[https://github.com/elastic/beats/pull/15712](https://github.com/elastic/beats/pull/15712) will probably help me out as well. Will try to see if I can build from source, or wait for 7.6.1 to come out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 23, 2020, 10:14am UTC](https://discuss.elastic.co/t/metricbeat-prometheus-debug-logging/220387/4 "2020-03-23T10:14:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.