Subtracting one value out of another and showing the percentage difference

Elastic Stack Kibana
1

I am trying to get a pie dashboard of successful vs abendent processes out of my logs.

My issue is that there is no way to filter the abendent processes.
So what I can do is filter based on the values that are successful and based on the total ones.

As it is logs they are no one entry, so I filter based on the logs of process start and process successful.

What I get is a pie that has the total vs the successful and what I want to do is make a formula that will give me the percentage of successful over the total - successful.

How can I do that using formula?

2

What version?
What Visualization? Lens?

Not quite clear...
Can you provide some samples / of your data and what you calculations are?

Perhaps You could use a runtime field to mark which ones are successful.

3

This is only part of them but as an example:

"log_id": "1234566788ab214",
"policy_id": "registration",
"time_lo_res": "1697182080000",
"@timestamp": "2023-10-13T07:28:27.545Z",
"action": “login_start”,
"session_id": "af443e6410c2",
"time": "1697182107545"
          
"log_id": "1234566788ab217”,
"policy_id": "registration",
"time_lo_res": "1697182080000",
"@timestamp": "2023-10-13T07:28:27.545Z",
"session_id": "af443e6410c2",
"action": “login_successful”,
"time": "1697182107745"
          
"log_id": "1234566788ab227”,
"policy_id": "registration",
"time_lo_res": "1697182080000",
"@timestamp": "2023-10-13T07:28:27.545Z",
"action": “login_start”,
"session_id": "af443e6410c2",
"time": "1697182108545"


"log_id": "1234566788ab427”,
"policy_id": "registration",
"time_lo_res": "1697182080000",
"@timestamp": "2023-10-13T07:29:27.545Z",
"action": “login_start”,
"session_id": "af443ee41ac2",
"time": "1697182108575"

Now in this case you can see that there are three login_start and one login_successful and I want to create a dashboard that will show me 1 successful and 2 abendent.

To filter those logs from all logs I use the action = login_start or action = login_successful filter but then I still need to use some formula to get the right values. There are no other attributes in the logs that I can use to make the distinction.

I am using version 8.11 and using Pie but Donut will also be fine.

4

Can you provide the actual calculation using the exampless above?

How are you identifying abedent? A start without a successful?

I am thinking you could leverage a lastest transform on the data as these data need to be correlated on session_id

This will just keep the latest state based on log_id and then the calculations are probably easier

5

Yes it is just start without a successful.

There are more logs in between the login_start and login_successful but for simplicity I only included those.

The data is already in elastic but now I want to generate a dashboard out of it. And it is not really user specific because before the user logs in there is no id and then the id changes. The session_id is just a number that increases based on the logs.

6

I highly recommend looking at the latest transform it is basically fit for purpose for your use cases. There is a UI builder in Kibana to create the transform

Kibana - Stack Management - Transforms

Otherwise you will struggle to do the analysis you're trying to do

Perhaps the session_id is the correlation...

7

My bad I do have a correlation it seems like I did not include it in my example. There is group_id that makes the correlation between the start log and successful log.

It is such that if the logs relate to one session they will all have the same value for group_id.

And I noticed that I can use the unique_count(group_id) to make sure I do not have duplicates but for now I get a pie that shows all logins vs successful ones while I want to get (total - successful) vs successful

8

Perhaps You can use KQL filter in that unique_count one with a filter "action": “login_successful” and one without a filter

What is your definition of total?

Keep working on the formula....

The latest transform would probably make this all easier

9

My issue is that when I do minus it will remove all the entries of successful

If I try something like:

unique_count(group_id) - unique_count(group_id, kql='action :"login_seccessful"')

I would like to somehow only subtract the count from the other case.

For the other case we can use the kql='action :"login_start"'.

10

typo

I do not know what that means you can do not logic

kql='not action :"login_successful"'

You can figure it out... I will say one last time the Latest Transform would probably take 5 mins to setup then you would have "processed / correlated" data to work with.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.