How to put a visualization directly in Kibana/ElasticSearch?

Hello everyone,

I tried something and I think it's worth sharing :slight_smile:

I'm trying to write a script to automatically create a new dashboard according to a specific user (for that, it is the search that changes).

More precisely, as a dashboard is made of visualizations, we need to create the visualizations in the first place. I was blocked at this step, creating the visualization directly via a CURL -XPOST request to ElasticSearch.

Because in order to do the final dashboard, we need to :

  1. Write the search(es)
  2. Create the visualization(s)
  3. Create the dashboard made of visualizations

Schematically :

Input : Username --> Myscript --> Dashboard of the user

To make things clear, a visualization is only a JSON document in the path (in my case, but surely in yours too) :
'http://localhost:9200/.kibana/visualization/*'
In ElasticSearch, and Kibana reads it to display it.

So it's simple, add a new visualization as you add a new document through the ElasticSearch API.
To do that, and to know what the visualization you want to create looks like, you can create it using Kibana web interface; and once you validate and it's added in ElasticSearch, you see the ElasticSearch document.

Let's say we created a Pie Chart using Kibana named "Test1" : we request it in ElasticSearch to see the document.

curl -XGET 'http://localhost:9200/.kibana/visualization/Test1'

You should have this kind of result :

 {
   "_index":".kibana",
   "_type":"visualization",
   "_id":"Test1",
   "_version":1,
   "found":true,
   "_source":{
      "title":"Test1",
      "visState":"{\"aggs\":[{\"id\":\"1\",\"params\":{},\"schema\":\"metric\",\"type\":\"count\"},{\"id\":\"2\",\"params\":{\"field\":\"type.raw\",\"order\":\"desc\",\"orderBy\":\"1\",\"size\":10},\"schema\":\"segment\",\"type\":\"terms\"}],\"listeners\":{},\"params\":{\"addLegend\":true,\"addTooltip\":true,\"isDonut\":false,\"shareYAxis\":true},\"title\":\"New Visualization\",\"type\":\"pie\"}",
      "uiStateJSON":"{}",
      "description":"",
      "version":1,
      "kibanaSavedObjectMeta":{
         "searchSourceJSON":"{\"index\":\"logstash-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"YOUR KIBANA DISCOVER SEARCH HERE\"}},\"filter\":[]}"
      }
   }
}

So, then, we want to create a similar document into ElasticSearch directly, without using Kibana, as we would need it in a script. The query looks like this :

curl -XPOST 'http://localhost:9200/.kibana/visualization/Test2' -d '
{"title":"Test2","visState":"{\"aggs\":[{\"id\":\"1\",\"params\":{},\"schema\":\"metric\",\"type\":\"count\"},{\"id\":\"2\",\"params\":{\"field\":\"type.raw\",\"order\":\"desc\",\"orderBy\":\"1\",\"size\":10},\"schema\":\"segment\",\"type\":\"terms\"}],\"listeners\":{},\"params\":{\"addLegend\":true,\"addTooltip\":true,\"isDonut\":false,\"shareYAxis\":true},\"title\":\"New Visualization\",\"type\":\"pie\"}","uiStateJSON":"{}","description":"","version":1,"kibanaSavedObjectMeta":{"searchSourceJSON":"{\"index\":\"logstash-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"YOUR KIBANA DISCOVER SEARCH HERE\"}},\"filter\":[]}"}}
'

Note that we used the Source part of the first query to make the POST.

That's it ! You can check the Pie Chart has been well created in Kibana :wink:

Have a good day, I hope this has been useful.

Tony

7 Likes

Thanks for sharing! I reformatted a bit your post so I hope it's even easier to read and reuse as a recipe.

1 Like

SoI think .kibana index is available only in version 4.1 or greater.
What about version 4.0.* or earlier?

I don't know I didn't test it and I'm not using an old version, but you can :slight_smile:

Hi!

I was trying to create the visualization directly, and I ended up with the same json object as you (templating in my own parameters where needed) The server returns 200, saying that the visualization is created, but it does not show up in the visualization list. Any clue why this might happen?

Response:
{"_index":".kibana","_type":"vizualization","_id":"test_viz4","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true}

A bit more information: I'm using kibana 4.4.2

The .kibana index takes visState which is a json string within a json object. The json string needs quote escaping and everything. Here is how to make a simple pie chart using term aggregation:
( note "New Visualzation". In 4.4.2 this is just a place holder it appears and MyTitle is what is important. )

curl -XPOST 'http://myserver:9200/.kibana/visualization' -d '{"title": "MyTitle", "visState": "{\"title\":\"New Visualization\",\"type\":\"pie\",\"params\":{\"shareYAxis\":true,\"addTooltip\":true,\"addLegend\":true,\"isDonut\":false},\"aggs\":[{\"id\":\"1\",\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"type\":\"terms\",\"schema\":\"segment\",\"params\":{\"field\":\"MyAggField\",\"size\":5,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}", "uiStateJSON": "{}", "description": "", "savedSearchId": "MySavedSearch", "version": 1, "kibanaSavedObjectMeta": { "searchSourceJSON": "{\"filter\":[]}"}}'

2 Likes

Exactly what I was looking for, but is there any suggestion on how to write a search?

Hi Peter,
I'm no more into elastic nowadays but by memory I'd say to look at the request through your web-browser analyzer ?
I would have done it like this imo
Good luck
Tony

ps: my bad, forget what I wrote.
If you look attentively on my original post :
I wrote in the 2nd quote :
{
"_index":".kibana",
"_type":"visualization",
"_id":"Test1",
"_version":1,
"found":true,
"_source":{
"title":"Test1",
"visState":"{"aggs":[{"id":"1","params":{},"schema":"metric","type":"count"},{"id":"2","params":{"field":"type.raw","order":"desc","orderBy":"1","size":10},"schema":"segment","type":"terms"}],"listeners":{},"params":{"addLegend":true,"addTooltip":true,"isDonut":false,"shareYAxis":true},"title":"New Visualization","type":"pie"}",
"uiStateJSON":"{}",
"description":"",
"version":1,
"kibanaSavedObjectMeta":{
"searchSourceJSON":"{"index":"logstash-*","query":{"query_string":{"analyze_wildcard":true,"query":"YOUR KIBANA DISCOVER SEARCH HERE"}},"filter":[]}"
}
}
}
Does it answer your question ?

Thank you for the info. This only answers how to create a visualization but does not give me an idea on how to create a saved search for the visualization to be used.

Use the content of the _source keyword you get from exporting a saved search and POST it to e.g. http://myserver:9200/.kibana/search or PUT it to e.g. http://myserver:9200/.kibana/search/ddddddd-dddd-dddd-dddd-dddddddd/_create