Visualizing sparse data in Timelion

[MPW]

I have 6 data points in a timeboxed search of data in an index:
@timestamp state
February 17th 2019, 07:15:01.000 1
February 17th 2019, 07:15:37.000 0
February 17th 2019, 07:19:29.000 1
February 17th 2019, 07:21:51.000 0
February 17th 2019, 07:49:40.000 1
February 17th 2019, 07:51:15.000 0

This data represents when the screen is turned on and off on an Android device.
I am trying to create a visualization that shows the screen on time as shaded areas of the graph (I will be superimposing other activities over the top).

The closest I've got so far is this:
.es(q='',index=devices-,metric=max:state).fit(nearest).lines(fill=1,width=0).color("grey")

There are two difficulties I am trying to solve;

First - most importantly for the visualization - is the graph before the first datapoint. This is the rule I need:
If the first datapoint is a '1' - then the initial line to that datapoint must be '0'.
If the first datapoint is a '0' - then the initial line leading to that datapoint must be a '1'.

I have no idea how to implement this... the .fit(nearest) is extending the area to the left with the actual value.

The second issue I have is the vertical lines defining the shaded boundaries are, well, not vertical. At the timestamp it becomes '1' - I need it to step vertically up to '1'. Likewise for the datapoint of '0' - it needs to step vertically down to '0'.

The final issue is the grey areas are being significantly widened. I suspect this is due to the bucket size - but that's not the whole story. 'Auto' the 0 to 0 and 0-1 transitions are often off by 30 seconds or more. If I set 1 second bucket sizes I get too many data points to represent. If I set 2 second bucket sizes the graph looks better (except the issues above) - but the transitions are still 10-15 seconds off. I'm not sure why....

I'm obviously missing something pretty fundamental - I feel like I'm swimming upstream with this... I just cannot find it in the documentation... Does anyone have any ideas about what I should try?

Thank you,

Michael.

[MPW]

I've found that .fit(carry) appears to solve the third issue - but I don't know why. Is there any documentation for what each mode does? Average and nearest are obvious - but the other options less so...

Still struggling zeroing the start... and with the transitions not being vertical (obviously - a screen can only be 'on' or 'off'.

Michael.

[bhavyarm]

@timroes can I please get some help here?

Thanks,
Bhavya

[timroes]

Hi Michael,

with regards to the fitting function, you most likely want carry. I think beside the auto complete documentation we're not having any good documentation on those values. I think you can use this issue which describes the fitting functions we want to have in all of Kibana as a reference of how different fitting function works. The ones existing in Timelion should have the same behavior as described in the issue.

To get straight lines, you most likely want to add the steps parameter to your .lines function: .lines(fill=1, width=0, steps=true). This will make this function a "stepped" function instead.

For the issue with the initial value: I don't think there is a solution to do that at the moment. I tried to figure out a solution, but since we're not having something like "lookahead" (as in the above fitting issue), I don't see a way how you could figure out "what's the next value" and then negate that, sorry

Cheers,
Tim