I have a string field (data.result) in my documents that can contain "SUCCESS" or "FAILURE". I'm running ~20 tests across ~50 servers and each test on a server has a "SUCCESS" or "FAILURE" result. I want to make a heatmap of the failure rates with test names on one axis and server names on the other axis. I am already doing this successfully with an average "test duration" field and it works great.
I think I just need to convert "SUCCESS" to 100 and "FAILURE" to 0 and then do the average of those to get a failure rate. I tried to convert the data.result field to numbers using the JSON Input field in the heatmap visualization like this:
{"script":"if (doc['data.result'].value = 'SUCCESS') { return 100; } else { return 0; }}
But I don't know which field I should select for the "average". I only have already existing numeric fields to choose from, not the result of this script. And it also gave me, "Courier Fetch: 25 of 34 shards failed."
I also tried to aggregate it in the script like this
{
"script":
{
"lang": "painless",
"inline": "int totalSuccess = 0; int totalFailure = 0; for (int i = 0; i < doc['_type'].length; ++i) { if(doc['data.result'][i] =='SUCCESS') {totalSuccess += 1;} if(doc['data.result'][i] == 'FAILURE'){totalFailure +=1;}} return totalFailure/(totalFailure+totalSuccess);"
}
}
This one gives me "Courier Fetch: 5 of 34 shards failed."
I also tried to add a scripted field to the index, but that gave me errors about too many script compilations per minute and we increased that and it just gave us similar errors to what we got above.
I'd also like to know what tools people are using to test/debug/validate their scripts before throwing them into production. It says to test the scripts out beforehand, but it doesn't say what tool to use for that.
I obviously don't know what I'm doing, so any help will be greatly appreciated. I have been a C++/Python developer for 25 years, but ELK is brand new to me.
If I can't get this working with a scripted field, I'll have to create a new field in the document that just has 0 for FAILURE's and 100 for SUCCESS's and then do an average on that numeric field. I'd really like to figure out how to get a scripted field to work in a heatmap though.