Serial Diff missing data

[lars.vanderspek]

Hi there,

When using the the Serial Difference aggregation in Time Series Visual Builder, I stumble upon missing results in the visualisation. For example, I use a time interval of 1 year to compare the sum of sales data per customers between two years. If a customer purchased goods in the first year, but didn't in the second year - the difference always seems to be zero. I would like the difference to give a negative result. The same is true if the the buyer only started buying in the second year. Also a difference of 0, where I would like to see a positive difference.

I looked at the reference and noticed an option that inserts zeros in gaps. However, in TSVB this option is not available.

Edit: I'm on Elastic Cloud 6.2.0.

Is there any way I can calculate the desired difference? Any help is appreciated.

[tsullivan (Tim Sullivan)]

Using Kibana 6.2.2, I was able to use the Serial Diff aggregation to do what you are describing. Take a look at my screen shot for a simple test case:

[lars.vanderspek]

Thanks for your reply, Tim.

Your example is indeed a lot like the case I describe. Now imagine that a new OS would login in the next hour, causing the count in that hour to go up by 1. That count value would be compared to the number of logins in the hour before: 0 (or actually null). The difference in this example would be +1, however, a serial aggregation would give a result of 0 - due to the lack of previous entries.

Now I'm wondering how to display the +1 I'm looking for, since this value is a result that I'm very much interested in. Should I post preceding zero values for each term to Elastic, for example? Or is there some form of setting that I'm overlooking?

To give a complete overview I added an example from my own dataset. I have a buyer for which the described problem occurs. For the first time this buyer makes an order - creating the first record with a turnover of 32 (screenshot 1). In the second screenshot I calculate the serial difference and get a value of 0 - here I'd like to see +32. I added Screenshot 3 to show which settings are applied to the visualization.

Screenshot 1

Screenshot 2

Screenshot 3

[Marius_Dragomir]

What youy're looking for is the gap_policy param for the serial diff aggregation, but that doesn't seem to be exposed as an option for TSVB. You should enter an enhancement request for it in the Kibana repo.

[lars.vanderspek]

Thanks for your reply Marius. I had the feeling this was the case. I'll have a look at posting an enhancement request.