Hello,
I have a ML model for image classification problem running on Azure Kubernetes instance on python. I would like to visualize some model metrics like precision, accuracy, recall etc. on a kibana dashboard. With every image sent as a request with ground truth, the metrics are calculated. I can construct charts of metric scores (y-axis) vs image_number (x-axis) on azure ml workspace and now trying to have those charts (score vs timestamp maybe) on a Kibana dashboard. I have established a connection with the ELK stack and filebeat (Installed on kubernetes cluster) and able to discover all my metric scores output in the filebeat log messages.
I would like to know how I could capture my score alone and make a chart out of it. Is there a way to make my metrics indexed in Elasticsearch and then build a dashboard out of it? I also came across the Elastic APM and metricset feature (push custom metric to Elasticsearch) but I could not understand the end-to-end setup for my application. Please provide me some detailed information or sample code to push metrics that could help me.
ELK and Filebeat version: 7.17.3 with helm charts (Latest)
Thank you.