Two different controls to be applied on two separate visualizations on one dashboard

[rubic]

Hi,

I'm trying create dashboard which will allow users to compare two, the same graphs showing hourly jobs distribution break down by date.
The goal is to apply two, different dates on each graph using controls - I know I can do it manually on each visual, but this is no go scenario.

Tried to place the same data in to separate indexes, data views - trick with ignore filters and lucene OR filter did not help me as well.

Do you guys have any ideas on how I could tackle it?

[carly.richmond]

Hi @rubic,

Welcome back! Can you confirm which version of Kibana you are using, which visualization type you are using (Lens or other) and why the manual approach is a no-go option?

Let us know!

[rubic]

Hi @carly.richmond, thanks for reply!

I'm using Lens as visualization type, but it is not a must - I can recreate it to any other type if needed. We are currently on Kibana v 8.18.2

I simplified description of my case to maximum, just to make it as clear as possible. Target solution will contain 6 different visuals per both: baseline and reference date. Number of visuals implicates 12, different manipulations which would be over complicated for non technical recipients of the dashboard - that is why it is no go scenario in this case

I hope it is a little bit more clear now

[Tortoise]

Hello @rubic
Could you please confirm if you want to compare data for any specific range?

Like if we want to compare can use below logic :

Step 1 : Add similar metric for comparison -

Step 2 : for the second metric use below configuration , use timeshift as per your need 1d,1w,1m

So in same graph it compares count of today with yesterday.

Hope it helps.

Thanks!!

[rubic]

Thanks @Tortoise, unfortunately proposed solution won't solve my issue either.

My aim was to use controls - one on a "baseline" date field from index a, second on reference date field from index b. Sets of visuals are exactly the same, I would like to see baseline data on one set and reference data on the other one.
Tried many different things already including ignore filters, switching from kql to lucene combining / splitting data in indexes / data views etc.
Without success so far.

[carly.richmond]

Hi @rubic,

Have you considered a custom visualization using Vega if @Tortoise's suggestion or having separate visualization controls are no-go options ? That way you have full control over the visualization and can in fact have custom event handling interactions.

[rubic]

Hi @carly.richmond,

Unfortunately vega did not help as well.

This is a draft of my dashboard:

First 3 visuals from right are lenses, next one is vega, filtered by x_id: 20 in kql.
Bottom visualizations contains the same set of fields, but with "_bl" suffix and are filtered by x_id: 50. I can store both data sets in the same or different indexes if needed - tried both options + ignore filters with different scenarios.

When I'm choosing dates from both controls, all visual elements are empty, even though data is ingested:

I suspect that within this configuration of controls, "AND" operator is used in the queries. This could explain why I'm struggling at this stage.

Request from vega, taken from inspect:

  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "x_id : 20 or x_id: 50",
            "analyze_wildcard": true,
            "time_zone": "Europe/Paris"
          }
        },
        {
          "range": {
            "timestart": {
              "gte": "2025-05-02T14:09:22.837Z",
              "lte": "2025-05-03T16:45:22.285Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "should": [
              {
                "match": {
                  "x_id": "20"
                }
              }
            ],
            "minimum_should_match": 1
          }
        },
        {
          "match_phrase": {
            "x_date": 1746136800000
          }
        },
        {
          "match_phrase": {
            "x_date_bl": 1743717600000
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  },

Is there anything I could do with my setup to use controls and get results on both data sets within one dashboards?

[Tortoise]

Hello @rubic :

If my understanding is correct , your are looking for a dashboard with below logic -
example : on a single dashboard you need 2 controls (date1 , date2) there will be 2 views (view1 , view2) can be many more views
date1 > view1
date2 > view2
So date1 changes should only impact view1 & no impact on view2 similarly for date2?

Thanks!!

[rubic]

Hello @Tortoise
Yes, this my scenario - two views, each with separate control:
date1 > view1
date2 > view2
date1 changes should only impact view1 & no impact on view2 similarly for date2 - correct.
There will be only two views in my case

[Tortoise]

Hello @rubic

I tried to create 2 indexes A , B copy of say index C.
In A i renamed 1 field response-a & in B to response-b
Created 2 lens from separate dataviews (separate indexes/fields)
Added 2 different controls to filter response-a for A & response-b for B
When we filter on1 view say A below are the observations :

1. It filters at dashboard level & will show data in view A but as same field in not present in view B that will be shown blank
2. If the field is present in both the views than it apply filters for both the view.

Can we add drilldown to the individual views? by this it will redirect to discover/dashboard meant for that particular index only.

Or maybe complete dashboard in Vega as suggested by @carly.richmond

Thanks!!

[rubic]

Hi @Tortoise

Thanks for verifying my case. It seems like it behaves exactly the same way as on my environment.

Goal is to have visual comparison of processing batches during to different days, so drilldown won’t solve the issue.

For now, I have two visuals written in Vega but placed next to lens graphs. Vega visuals reacts to controls exactly as lenses. I could of course rewrite everything in Vega, if you suggest it could solve my case.

I’m also thinking about calculating number of seconds from midnight (x axis) and store this value in the field. Then I would count nr of documents over 3600 sec / 1h. I could use one control, with multiple choices, for two different dates and visualize it in Vega as bar chart, on one visualization.

As a last resort, users will have to use two, browser windows, but for obvious reasons I’d like to avoid this solution

[rubic]

Just tried option with vega visualizations - with no luck, it reacts on controls exactly the same as lens.

[carly.richmond]

Can you share your code and confirm which controls/ filters you have in the dashboard in which the Vega visualization is used?

[rubic]

Sure @carly.richmond

My dashboard consists of two vega visuals and two controls:
Control 1, field X_date -> View, vega 1 / X1 index
Control 2, field X_date_bl -> View, vega 2 / X2 index

Vega code is Identical for both visuals, result set of the es query contains the same set of fields, X2 index fields have _bl suffix.

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "url": {
      "%context%": true,
      "%timefield%": "time_start",
      "index": "X1*",
      "body": {
        "aggs": {
          "per_hour": {
            "date_histogram": {
              "field": "time_start",
              "calendar_interval": "1h"
            }
          }
        },
        "size": 0
      }
    },
    "format": {
      "property": "aggregations.per_hour.buckets"
    }
  },
  "mark": "bar",
  "encoding": {
    "x": {
      "field": "key_as_string",
      "type": "temporal",
      "axis": {"title": "Time (hourly)"}
    },
    "y": {
      "field": "doc_count",
      "type": "quantitative",
      "axis": {"title": "Document Count"}
    },
    "tooltip": [
      {"field": "key_as_string", "type": "temporal", "title": "Hour"},
      {"field": "doc_count", "type": "quantitative", "title": "Documents"}
    ]
  }
}

[rubic]

In the mean time I managed to implement scenario I was describing in the post with @Tortoise

I've converted start_time into nr of seconds and counted nr of documents over 3600 sec / 1h intervals

It works a little bit differently but does the job:
Control 1 with multiple selections, field X_date -> View, lens / X1 index

Maybe it will be helpful to someone with similar task to mine.