Percentages per day (per bucket) possible?

Elastic Stack Kibana
1

I have a set of records where each record has (among other fields) a field "spam" for the amount of spam mails received and a field "total_volume" for the amount of mails, received. For each day I have several entries for different mailboxes.

So the data looks something like this:

POST _bulk
{ "index":{"_index":"test"}}
{ "spam":1, "total_volume":   2, "mailbox": "mb1", "date":"2019-01-01"}
{ "index":{"_index":"test"}}
{ "spam":1, "total_volume": 100, "mailbox": "mb2", "date":"2019-01-01"}
{ "index":{"_index":"test"}}
{ "spam":2, "total_volume":   2, "mailbox": "mb1", "date":"2019-01-02"}
{ "index":{"_index":"test"}}
{ "spam":2, "total_volume": 100, "mailbox": "mb2", "date":"2019-01-02"}

I'd like to get a visualization which would show me about 2% for 2019-01-01 and about 4% for 2019-01-02.

What I could achieve, using JSON input

{
"script": {
        "lang":   "painless",
        "inline": "_value * 100 / doc['total_volume'].value"
      }
}

was the average per day, but this results in too high values. For example on 2019-01-01 the high rate of mb1 (50%) is averaged with the 1% of mb2.

So the result is 25.5% for 2019-01-01 and 51% for 2019-01-02 instead of the required 2% and 4%.

I have no clue how I could achieve this.

Example what I used to get the results as a table:

image
image.png626×1068 36.9 KB

image
image.png630×972 41.2 KB

2

You can try with TSVB and the Math aggregation. Then you should be able to calculate the percentage of spam as "100 * sum(spam)/sum(total_volume)".
Basically create 2 sum metrics, one for spam, one for total_volume.
Then create a math aggregation with the equivalent of the formula that I added above, and then keep just this line visible on the chart.
PS: don't forget to keep the bucket size to 1d.

3

Thanks for that hint. I already tried it and tried again now. I get a strange error message

[tsvb] > No reason phrase

image
image.png836×811 47.6 KB

4

It gets weird now…

If I put as Expression add(multiply(params.sum_spam,100) , params.sum_total) I get a graph.

If I replace "add" with "divide" => No data to display…

5

can you try with mathematical signs instead of the functions?

6

Made no difference, but I managed to solve the issue by using "Bucket script" instead of "Math".

So issue solved thanks to your hints.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.