Dynamic Dashboard Schema and Dynamic kibana charts

Elastic Stack Kibana
1

Hi All,
I trying to find the way to configure/develop kibana Dashboard that change dynamically over time.
I mean that i stream 100 parameters data into elasticsearch in real time and want to see the top 10 parameters every 5 minutes.

  1. How i can show different top 10 parameters in the same chart every 5 minutes?
  2. How i can show different top 10 parameters in separate 10 charts every 5 minutes? How to enforce reconfiguring of kibana dashboard DYNAMICALLY ?

Thanks in advance,
Really appreciate any help
David,

2

You can enable the "Auto-Refresh" feature of a Dashboard so that the data is re-fetched at an interval you specify. You can make it every 5 minutes if you like.

enable auto refresh.gif1443×571 294 KB

3

Hi Tim Sullivan,
Thank you about your response.
It seems that I found the solution to that problem (but not automatically and safely as i expect -any suggestion are highly welcomed):

  1.   How to Import/Export  dashboard as json file manually:

You can export saved dashboards, visualizations, and searches from Settings >> Objects (see below)
Remember to export associated visualizations and searches alongside the dashboard...
just clicking on dashboard export does not automatically include dependent objects.
2. Kibana under the hood: object persistence:
Users of Kibana create saved searches, visualizations, dashboards, and other such objects.
Kibana, of course, has to persist these objects somewhere so they can be loaded up next time users start it up.
As Kibana connects to Elasticsearch for querying users’ data anyway, it conveniently uses Elasticsearch to store its objects as well.
This blog post explores some of these objects, how they are stored in Elasticsearch, and when they are created.
The goal is for this information to serve as a useful tool not only for debugging but also for administrators who might want to deploy Kibana in an automated, repeatable fashion today.
3. Creating Kibana Dashboard Programmatically:
You can either write directly to the .kibana index or use the URL structure to create dynamic dashboards (though the latter requires you already have saved visualizations you're combining).
4. Building Kibana Dashboard Dynamically:
Look at the .kibana index in your elasticsearch ... it contains all the visualizations , dashboards etc.
You can use the normal elasticsearch REST Api to modify and add to this index.
Alternatively , a lot of what drives a dashboard (the different panels and filters) can be changed directly in the url (in RISON format)
generate the dashboard object with graphs for each new chart and then push it to the .kibana index ,
which will then update the existing dashboard and everyone who uses it will get the updated dashboard next time they refresh.
The visualizations are also stored there , so the same concept applies where you would first generate a visualization object
(tying to extract the data to the specific new chart from ES before )
to push to .kibana , before you reference it in your dashboard.

1 Like
4

Thanks David
I had been wandering around for an answer to this question and I could not find this simple solution anywhere.
It worked for me very nice.

Siamak

5

Hi,

This is very useful information for creating dashboard and visualizations dynamically.

Is there a way to dynamically generate the panelsJSON field?

"panelsJSON": "[{"col":1,"id":"Active-Requests-By-Server","panelIndex":7,"row":9,"size_x":6,"size_y":4,"type":"visualization"},{"col":7,"id":"EL1-Cache-Hits-By-Server","panelIndex":8,"row":5,"size_x":6,"size_y":4,"type":"visualization"},{"col":1,"id":"L1-Cache-Hits-By-Server","panelIndex":9,"row":5,"size_x":6,"size_y":4,"type":"visualization"},{"col":7,"id":"Stream-Count-by-Server","panelIndex":10,"row":1,"size_x":6,"size_y":4,"type":"visualization"},{"col":1,"id":"Total-Stream-Count","panelIndex":11,"row":1,"size_x":6,"size_y":4,"type":"visualization"},{"col":7,"id":"Queue-Depth-By-Server","panelIndex":12,"row":9,"size_x":6,"size_y":4,"type":"visualization"}]"

Currently, the panel should be defined with the row, column information.So, everytime we need to add/remove a visualization, we need to go through existing list of visualizations in the panelsJSON field and add/remove our visualization. Instead it would be nice if there is a api call for adding visualization to the existing dashboard.

6

Hi Raji,
Currently there are no APIs in Kibana for what you are trying to do.

7

Thanks, i ended up writing a program to dynamically add/remove visualizations in dashboard.

8

Nice! That's the recommended way to manage this.

9

Hey,
Can you please share how did you do this?

10

Hey Ashvita,

Sorry for the delayed response. I ended up writing a go program which connects to elasticsearch using the net/http client, creates new visualizations, fetches the existing dashboard, adds the new visualizations to the dashboard and uploads the new dashboard in elasticsearch.

For example, lets say you have a dashboard called Network-Bandwidth and need to add ingress and egress visualizations of your servers in it.

  • Fetch all visualizations present
    GET - http://localhost:9200/.kibana/visualization/_search

  • Check if the visualizations you need exists, if not, create new ones
    PUT - http://localhost:9200/.kibana/visualization/Egress%20Bandwidth%20-%20abc
    {
    "title": "Egress Bandwidth - abc",
    "visState": "{"title":"Egress Bandwidth - abc","type":"timelion","params":{"expression":".es(index=\"logstash-\", q=\"system.network.type.keyword:(Egress OR Both)\",metric=\"max:system.network.out.bytes\", split=\"system.network.name.keyword:20\", kibana=true).derivative().multiply(8).scale_interval(1s).if(operator=\"lt\", if=0, then=0).trim(start=2,end=1).label(regex=\"^. system.network.name.keyword:(.+) > .$\", label=\"$1\").lines(width=2).yaxis(units=bits/s, min=0)","interval":"auto"},"aggs":[],"listeners":{}}",
    "uiStateJSON": "{}",
    "description": "",
    "version": 1,
    "kibanaSavedObjectMeta": {
    "searchSourceJSON": "{"query":{"query_string":{"query":"    "}},"filter":[]}"
    }
    }

  • Fetch the dashboard using the following URL
    GET - http://localhost:9200/.kibana/dashboard/Network-Bandwidth/_source

  • Re-create the panelsJSON of the dashboard including the new visualization(s)

  • Upload the new dashboard using the following
    PUT - http://localhost:9200/.kibana/dashboard/Network-Bandwidth
    {"title":"Network-Bandwidth","hits":0,"description":"Graph with network bandwidth for nodes","panelsJSON":"[{"col":1,"id":"Egress Bandwidth - abc","panelIndex":2,"row":1,"size_x":6,"size_y":4,"type":"visualization"},{"col":7,"id":"Ingress Bandwidth - abc","panelIndex":8,"row":1,"size_x":6,"size_y":4,"type":"visualization"},{"col":1,"id":"Egress Bandwidth - test123","panelIndex":7,"row":6,"size_x":6,"size_y":4,"type":"visualization"},{"col":7,"id":"Ingress Bandwidth - test123","panelIndex":13,"row":6,"size_x":6,"size_y":4,"type":"visualization"}]","optionsJSON":"{"darkTheme":false}","uiStateJSON":"{}","version":1,"timeRestore":true,"timeTo":"now","timeFrom":"now-30m","refreshInterval":{"display":"30 seconds","pause":false,"section":1,"value":30000},"kibanaSavedObjectMeta":{"searchSourceJSON":"{"filter":[{"query":{"query_string":{"query":"*"}}}],"highlightAll":true,"version":true}"}}

2 Likes