Kibana Lens/Line customizations

[GustavoEH]

Hello,
I created a Line view in Kibana to see the data I'm getting from different exchanges, but I' having difficulties to customize the view. As I don't have much experience with Kibana, now I'm not sure if Lens/Line is the best way to visualize the data the way I need.
This is what I have so far:

I have 3 exchanges with ask and bid prices.
What I need to get a perfect view of the data (these are the properties I couldn't adjust):

• Solid lines. Doesn't matter if is missing data, dotted lines does not help this view;
• No points. That big circles also doesn't help in this case;
• X-axis resolution. My data have an interval of approximately one second, and I need to see this raw data, no aggregations, even when I'm trying to se one hour interval;
• Same color for the same exchange. Ask and bid prices from the same exchange must be the same color. If not automatically I can do it manually.

Is it possible do customize a Lens/Line this way? If not, is there another way to view the data with all these customization? Maybe TSVB?

Thanks in advance.

[Tomo_M (Tomohiro Mitani)]

Hello,

In my opinion kibana is designed for (possibly almost specialized) visualize aggregation. To visualize individual raw data seems not to be its main scope. You have to use some trick to implement such chart. One typical trick is using fine interval such that each bucket contains up to one documents.

You have detailed customization requirements, so I don't think Lens, TSVB or aggregation-based line chart will meet the requirements. I'm not sure if it is feasible, but one possible way is timelion and another way is Custom visualization using vega-lite.

Timelion

.es(index=kibana_sample_data_flights, timefield=timestamp,
    q=DestCityName:Moscow,
    metric=avg:AvgTicketPrice)
    .lines(fill=1,width=0.5)
    .fit(mode=average)
    .lines(fill=0)
    .color(green),

.es(index=kibana_sample_data_flights, timefield=timestamp,
    q=DestCityName:Quito,
    metric=avg:AvgTicketPrice)
    .lines(fill=1,width=0.5)
    .fit(mode=average)
    .lines(fill=0)
    .color(green)

Custom Visualization

With Custom Visualization, you don't have to use aggregation. You can use any elasticserach query. You can integrate dashboard context / time filter with your query using special tokens such as '%timefield%: <name> or %dashboard_context-must_clause%.

{
  $schema: https://vega.github.io/schema/vega-lite/v5.json
  title: Event counts from all indexes

  data: {
    url: {
      index: kibana_sample_data_flights
      body: {
        size: 10000
        query:{
          bool:{
            must: [
              "%dashboard_context-must_clause%"
            ]
            filter:[
              "%dashboard_context-must_clause%"
              {
                range:{
                  timestamp:{
                    "%timefilter%": true
                  }
                }
              }
              //just for avoid overcrowdedness in thi sample
              {
                terms:{
                  DestCityName: ["Shanghai"]
                }
              }
            ]
            must_not: [
              "%dashboard_context-must_not_clause%"
            ]
          }
            
        }
      }
    }
    format: {property: "hits.hits"}
  }

  mark: {
    type: line
  }

  encoding: {
    x: {
      field: _source.timestamp
      type: temporal
      axis: {title: false} // Customize X axis format
    }
    y: {
      field: _source.AvgTicketPrice
      type: quantitative
      axis: {title: "Document count"}
    }
    // multiple lines for DestAirportID
    detail: {
      field: _source.DestAirportID
      type: nomial
    }
    // multiple lines colored same by DestCityName
    color:{
      field: _source.DestCityName
    }
  }
}

P.S. these are just for my practice, so there may be some parts that are not good.

[ghudgins (Graham Hudgins)]

I would try to play with the time interval control on the "@timestamp" field. you can override and get a more granular time series. However, this does impact performance and there are some max buckets advanced setting configuration to mitigate a chart from being too verbose. Your last value can get the latest value based on some sub aggregation within the selected time interval. Commonly it can be good to selected the latest value based on timestamp but it'll all depend on your data.

We don't yet support document-level visualizations in Lens...but it's something we're interested in adding in the future. You may need a custom visualization if you want every document to be a data point. This is very similar to Maps so the Visualization would have to fetch 10,000 documents (or so) and display them.

On the styling point, I logged an issue for these controls you requested. Feel free to comment if I didn't capture appropriately!