Counting string occurrences in string

[gregpaskal]

I'm looking for a way to count the number of string occurrences within a string.

Say for instance I have the following string "Hello World" and I wanted to return the number of "l" found within this string. How would I script this in "Painless"

Thanks,
Greg

[Brandon_Kobel]

@gregpaskal where are you wanting to use this information, are you intending to build a Visualization using this information, and is this a set "string" that you're looking to count the occurrences of?

[gregpaskal]

@Brandon_Kobel, happy to have your response. You are correct, I will be trying to build another visualization, similar to the last one you assisted me with. Yes, I will be searching for a string (keywords) within a field called payload.test_case_exceptions.

What I'm trying to accomplish:
Raise awareness through a counter/metric/visualization when certain keywords show up in the payload.test_case_exceptions field.

Details:

1. I have a field called "payload.test_case_exceptions".
2. Anytime a test case fails, verbose details pertaining to the failure are recorded in the payload.test_case_exceptions field.

Example data recorded in payload.test_case_exceptions field upon failure.

[FAIL] 1 of 3 - Not Exist ID:003 - [server] found within [Your browser sent a request that this server could not understand.]
[FAIL] 2 of 3 - Not Exist ID:003 - [server] found within [Size of a request header field exceeds server limit]
[FAIL] 3 of 3 - Not Exist ID:004 - [Bad Request] found within [400 Bad Request]

Anytime "server" is recorded in the payload.test_case_exceptions field, I want to call attention to that on dashboard, through a new visualization dedicated to errors that require higher priority attention. I have a number of keywords or phrases (like "server" "Bad Request") I'd like to search for in the payload.test_case_exceptions field.

What I've tried to date:
I initially tried to create a scripted field that would perform this type of functionality and record a count of how many times a keyword (server, Bad Request, etc). was found in the payload.test_case_exceptions field but it was simply too much overhead for our elastic configuration and I got all sorts of shard errors.

Where I think I should explore next:
I think I'd like to approach this challenge with Visual Builder again, this time, trying to figure out how it can be used to identify these keywords (server, Bad Request, etc) within payload.test_case_exceptions. This would make it easier to add new keywords of phrases that I want to raise awareness to in the future.

Any help or direction @Brandon_Kobel would be greatly appreciated.

Greg

[Brandon_Kobel]

Thanks for the detailed description @gregpaskal. You could potentially use a scripted field to do this, but you have a couple other options as well.

Is the payload.test_case_exceptions field text or a keyword, or both? If you're storing the payload.test_case_exceptions as a keyword, then you can use a a filter aggregation similar to the following to bucket all of these exceptions similar to the following:

If we're dealing with a really large amount of data, this isn't going to be the fastest approach, and I'd suggest using Logstash and the grok filter to extract this logic into a boolean field, which will make this a lot more scalable.

[gregpaskal]

@Brandon_Kobel, I had success yesterday actually following much of what I learned in the previous visualization aggregation. The conciseness of the visualization is awesome, packing in a lot of critical information into a small space. From the dashboard that contains these metrics, I can fairly quickly derive where we have higher risk and where we have an acceptable level over a weeks worth of automated testing results.

Thanks again for the coaching. I have my next project in mind based upon yesterdays results.

Greg

[Brandon_Kobel]

@gregpaskal that's awesome, your detailed and clear descriptions of the problems you're trying to solve make our interactions productive and enjoyable!

[gregpaskal]

Thanks for your help @Brandon_Kobel.