We've been able to identify the correct Painless script for our scripted fields and the correct metric type (average) to display the data the way we want to see it. The problem we're still having is that since our docs don't always have the field we're calculating on in the scripted fields, the gauge visualization (we settled on that vs. the pie chart) is still counting those docs in its average calculation. So even though 100% of the calls to a certain endpoint may be getting 404s, the gauge is still only showing 33% because only a third of the docs in this index actually have information about that endpoint.
I've modified our scripted fields so that they return null instead of 0 when the "total_count" field mentioned below is empty, but how do I then exclude it from the gauge? We have lots of fields that we could use to filter, but I haven't found any meaningful information on the JSON Input field in the gauge chart to show me how to leverage that. Any ideas?
I'm somewhat new to the ELK stack, so let me preface this by saying that if someone can point me to the right section in the docs to find my own answer, I would value that just as well as someone saying "Do it this way".
We are using Elasticsearch to index performance messages from our services. We had to switch from sending a message for each call to sending a single message every minute with rollup metrics from the services due to traffic issues. We're trying to update our visualizations to match.
The problem that we're running into is that for some of these visualizations (namely pie charts), we're struggling to wrap our brains around the necessary steps to express the ratio of two separate values in a single pie chart, as opposed to just the counts for each value over a set of documents. To be specific, we now have the number of 3xx responses and 4xx responses as separate fields in each document, whereas we used to just count the number of documents with a 3xx response code vs those with a 4xx response code.
I'm certain that there's a way we can use Painless or even some simple JSON to achieve this, but I'm struggling to figure out how. Can anybody point me to the information I need to figure out the answer?