When using log scale in line chart with Y values = count, there are two problems:
-
Log scale does not work for values 0
-
The error message is odd telling that it does not work because of values lower than 1.
-
I still need a log scale for my non-zero values so I would suggest to have a specific output when input is 0, for instance by not drawing only the points for this 0-values inputs so I could at least see the rest of the graph.
-
It seems log scale is not available for values lower than 1 and greater than 0, why? Can't you draw negative values on Y-axis?
Thanks
To start with, you can't plot negative values on a log scale. There's a mathematical reason for this, as log values of negative numbers are imaginary. This explanation is pretty good.
Log of 0 has a similar problem, but instead of being imaginary, it simply doesn't exist. The same site has a good explanation of that as well.
As for values between 0 and 1, they are effectively indistinguishable from 1 on that scale. There's a technical reason that plotting those values is impossible, but I don't recall what it was... it may have to do with the fixed sizing of the y-axis that we use though.
The short of it is that you probably shouldn't be using a log scale to visualize the values you have.
Thank you for your answer.
I have a count value which is 0 for some x and requires a log scale for some others because some values are in the 10s while some others in the 100s and some others in the 1000s, this is typically a case where log-scale is useful to see if an exponential relation exists between y and x. In another case I have very small positive values below 1 but still positives and very high ones. This is why I suggest Kibana can deal with this by plotting negative values in y for values between 0 and 1 in x, and not plotting for value 0 in x but still plotting the rest.
I suppose it's possible that we could just drop 0 values form the chart, but then the data won't really be accurate. As for values between 0 and 1, I'm not sure what the solution is there.
I'd encourage you to open an issue about it, and include information about your use case (much like you have here) and some sample data to help make your point.
@Michael_Aupetit: Did you open an issue on this? Do you have a link to it?
@Joe_Fleming: Let's assume I am plotting log messages counts on a bar-chart and split those bars by message severity. On some days, I may get a handful of error messages. For the sake of argument, let's say it's less than 10 messages. Some other days, I get no errors at all. Then something bad happens and I get a few thousands of messages. While I'm interested to see the spike in messages, this effectively "squishes" down the days with a only a few messages down to a few pixels, making them unreadable. Using a log scale would help for this. But as soon as there's a bucket with 0 messages, the whole chart is replaced by the errors message discussed in this topic. This makes the log-scale totally useless for this use-case. What would you recommend to use in such a case?
That's a hard question to answer. The log scale would in fact help dramatically with the "squishing" problem (a very common problem). I'm certainly not the strongest when it comes to logarithmic math, so I'm not sure what makes sense here. Maybe adding an option to ignore 0 values makes sense? Maybe not?
@thomasneirynck do you happen to have any insight here? What makes sense when trying to use log scales and dealing with buckets that have 0 values?
hi @Michel_Albert @Michael_Aupetit
As for not showing negative values, that is because we use log-scaling for squashing purposes primarily. Drawing negative values would defeat that purpose.
But that the chart now shows nothing when there are values below 1, is not ideal, I agree.
A current-work around would be to create a scripted-field that 'justifies' all values to at least be '1'. That might work for display purposes, but obviously would be fudging the data somewhat.
As @Joe_Fleming suggest, I would open an enhancement request in the Kibana repo. That will give the issue visibility as well.
I think the best solution for Kibana would be not to use log-scaling for squashing purposes, but a better easing function that can deal with values in the 0-1 range (or even negative values, although it is harder to wrap your head around what exactly the behavior should be in that case).
Done: https://github.com/elastic/kibana/issues/11723
Also added a screenshot for this particular case from one of out production dashboards which illustrates the problem quite well.
1 Like
Hello
I unfortunately had to change project so no more working with Kibana.
log(0) is -infinite still log is a way to change power in products, and
products in addition so I don't see any otgher funciton doing that.
To cope with log(0) or log of a very small value that the machine cannot
handle, I suggest having a specific color code to signal here is something
special. If the bars cannot be negative at all, because the x in log(x) is
always >=1 (as a count), then do the same (color coding in grey for
instance a bar with unit height or minimal visible height) for any value
x<1 and explain this in a tooltip when hover on the bar: "The count value
is <1 in log-scale".
Hope this help
Best
Michael
thanks @Michael_Aupetit for the update, and your explanation why this is a dealbreaker for you. I'd recommend you upvote the issue https://github.com/elastic/kibana/issues/11723, so you can follow up on it and be notified when it should get resolved. Thanks for the suggestion too on using the special color-code to indicate 0-1 values.