Dec 20th, 2022: [EN] State of the art in Data Visualization at Elastic

Data visualization is a search process

A visual search can be considered a perceptual investigation of visual differences between in the presented information like a standing out geometry over a hundred similar shapes, a slight change in the line curvature, or a closed set of neighbor points: in this visual space, your eyes are the search engine, the most powerful one (except for Elasticsearch of course :D). But to speed up and sharpen such visual searches we need to carefully craft our visual experiences. Here are some of those small, mostly unconscious, details that we introduced in the last year to improve your visual searches.

Multi-layer time axis

Time is not one of the visual dimensions, but we still perceive it as a steady, mostly linear, progress of events. Time is a natural cadence of states (night/day, seasons, moon phases), a physically perceived trend, and it deserves the same importance it has as in our daily routines.

A while back we decided to "invest in time" and we ended up releasing in Kibana 8.0 an improved time axis used on our time series charts. We wanted to lower the readability barriers by orienting with temporal landmarks and inducing a predictable and sensible cadence of time.

Here are some highlights of such time axis:

  • left aligned tick labels: this brings the forward(right)-moving concept of time into the axis. The tick shows the starting point of a day, an hour, a minute giving the idea that starting from this tick to the next one can be considered a time bucket.
    It also removes the need of shrinking the chart width due to overflowing labels near the chart edges.

  • temporal raster hierarchy: year, month, weeks, and days are hierarchical concepts that help you understand the cadence of time. Using a non-linear increase of gridlines saliency helps distinguish those time markers and increase the predictable and sensible cadence of time.

  • keep the time context around: Even if we don't show the entire timestamp on every tick label, the entire date is easily reconstructed by concatenating the lower layer with the topmost. This will help keep the time/date context around: you will never lose which month/day/year you are looking into.

  • reduced/limited repetitions: we reduced to the minimum required number of text repetitions, reducing the visual noise and providing only the essential time information that allows you to scan quickly through time

  • improved time formatting: the time formatting is internally configured depending on the number of displayed labels and the available space.

Discover histogram, Lens, TSVB and most of the time-series charts already are using the new time axis. The previous version is already deprecated and will be probably removed soon.

A new Metric chart

Metric charts are used to convey a single value metric. Showing numbers alone can be done in multiple ways, but a metric chart can give a bit more context and color to them.

A metric chart doesn't live alone but with other metrics and charts in dashboards and pages. Visual search needs consistency to reduce the burden of quickly scanning through multiple numbers, text, and colors. This leads us on looking for better-aligned design for our metrics in Kibana.

Screenshot 2022-12-16 at 17.45.58

The new metric introduces some sort of styling rigidity in favor of an improved consistency across the myriad of use cases such a chart could handle.
Here are some highlights:

  • right-aligned numbers: improves the comparability of multiple numbers vertically placed (as in tables). Centered or left-aligned numbers are more difficult to compare across panels, as digits don't align on the decimal point, forcing the reader to visually count the number of digits to be sure which number is bigger than the other. This is clearly a micro-optimization, but when applied to a relatively high volume of numbers it can definitely improve the readability.
  • break-point font size: font size doesn't scale linearly accordingly to the panel width/height, but use 3 different size breakpoints to adapt accordingly. This reduces the visual differences that are visible on some dashboards where low-digit numbers or slightly bigger panels are shown visually bigger values than others, the font-size shouldn't describe the magnitude of the value, particularly when such numbers are not part of the same numerical domain.
  • visual enhancers: if a metric has a semantically meaningful maximum, then a bar can represent the current value relative to that maximum. If the metric can vary in time, then a background trend can be used to represent it. Those are just indicators and eye candy used to improve a bit the magnitude perception of the visualized metric.

WebGL2 Flame graph

Elastic Universal Profiling is using a new improved implementation of the flame graph using the latest WebGL2 standard.
Taking the advantage of writing our own set of shaders (without any external library dependency_ and minimal data manipulation on the client side, we can now render from 0.5 million to a few million nodes on common hardware rectangles with a smooth 60FPS for interactions and animations.

ezgif-4-641351e64c

The nodes are WebGL rendered, with styling done in shader code (gaps, zoom invariant corner rounding). The Zoom/pan is via setting uniforms (providing a cheap and performant way to interact with the chart).

Zoom, pan, and search use animation tweening to maintain context-aware navigation.
A small portion of the left/right neighbor nodes is kept around the currently selected/focused tree node to maintain a sense of context. The minimap will also give you an overview of the overall tree, highlighting the current section in focus.

From/to state are both representable in the input data (array attributes), so it can tween:

  • incremental query response delivery
  • periodic data update
  • the user reorders nodes or uses a different aggregation logic

Geometry picking (detecting what rectangle the user clicks/hovers over): we used the same geometries rendered into a texture, with unique color coding for a bijective relationship to the node index, using a simplified fragment shader (oblivious to rounding/gap)

The node label texts are currently rendered as a Canvas2d overlay, as it preserves all formatting options, and there’s never any occluded text rendering. Only 100 to 1k nodes will ever be visible on the screen so this method is good enough for now. In the future, we might consider blitting Canvas2d content to the render buffer, or SDF/MSDF fonts etc.

We keep improving your search process

These are just some of the little internal details that we believe will improve your visual searching experience. More will come in the following months, from a refined tooltip and the possibility to do in-chart actions to an improved and aligned design across all charts.
New visualization will be available in the next feature to deliver new and improved ways of exploring your data.
Have a nice search day :smiley:

9 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.