I am trying to find a way to expose beats (especially the metricbeat) statistics to prometheus to have the option to monitor the correct functionality of metricbeat in terms of its connection to elastic.
Currently we observed, that in case of disconnection of pods with elastic, the issue is visible in the pod logfile, but this is silent - pods stays running, so the issue is not detected.
Idea would be to detect either the errors directly by prometheus, or alternatively monitor data flow (eg create event when processing is stuck / buffering ...).
Please is there anything you could you eventually advise to use for this scenario?
I believe, the additional metricbeat instance will allow sending the information to elastic, but I need to have the issue with metricbeat transformed to the Prometheus alert...
In the meantime I managed to to expose the stats/state json statistics using the http endpoint
In case of
running metric beat in standalone mode via configuration
as docker container using override "-E http.enabled=true" when running it
and same was achieved when running the metricbeat as kubernetes pod - but here I was expecting the metricbeat configuration propagated to pod via config map (http.enabled: true) would be sufficient, but it was ignored at all, so had to again update the calling the docker to make it work...
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
Welcome to the forum