Any new possibilities to display raw, un-aggregated time series data in kibana?

[defalt]

Hello ,

I asked the same question about 2 years ago and only found the same questions (and answers) in the forum from around 2018-2019 so I want to ask if something changed with the new Kibana and Elastic versions.

1. Is it possible to show unaggregated raw data in Kibana? We have timeseries data with simple numeric values like [5,15,15.2,19,12] and want to display them as they are, no average, count, sum etc. Is this possible by now (without Vega, because thats just ...) or is there still no way to do this properly? We know its impossible to display millions of raw data points, but kibana starts averaging with minimal points even though I think it could handle a couple thousand points.
   We worked around this issue by building our own application that does the visualization but if by now kibana is able to do this that would be great.

2. If it is possible does the solution also allow to display the data from multiple indices? We want to display the data from 5 indices in 1 visualization. The data has exact timestamps and only numeric values so I guess it should be possible but dont know.

Thanks for any infos on this topic

[flash1293 (Joe Reuter)]

We are working on this. It won't be released in the very next version, but it's a high priority on our roadmap.

Till then:
The high-level editors (non-vega) still always aggregate your data, but you can see this more of a way to save you from accidentally loading millions of data points and putting your whole system under heavy load.

but kibana starts averaging with minimal points even though I think it could handle a couple thousand points

You are absolutely right, it can handle a couple thousand. You can set the advanced setting histogram:maxBars to configure at what point Kibana will start scaling up your buckets. 100 is definitely on the low end, put I wouldn't advise going beyond 5k or so.

This allows you to essentially show unaggregated data, with the safety net of preventing extreme numbers of data points.

For better performance, I recommend disabling "include empty rows" in the date histogram definition and configuring the "missing values" option as "line"

Otherwise Kibana will process more data than necessary, and for multiple thousand time buckets it will start to matter at some point (especially on a dashboard together with other charts).

If it is possible does the solution also allow to display the data from multiple indices

Yes, Lens allows to do that by adding multiple layers, see this step-by-step guide: Analyze time series data | Kibana Guide [8.11] | Elastic

[defalt]

@flash1293

It won't be released in the very next version

With version do you mean it wont be released in 8.4 or do you mean it wont be released in 9.X?

Thanks for the information, looks like we will have to wait some time till we can stop updating our own application for this use case.

Thanks again!

[flash1293 (Joe Reuter)]

It won't be in 8.4, it's definitely not as far away as 9.x, but it's hard to give a specific version.

I hope the workarounds help for your use case.