When writing a Kibana plugin (especially a visualization) you sometimes need to make custom ES queries, i.e. queries, that do not use the aggregations, that you can define via
Schemas in your visualization definition. You might still want to have these integrate with the set filters, queries auto-refresh times, etc.
An example: I have an index, that contains tweets from twitter. I would like to create a visualization, that shows a random tweet from "the result". For the sake of the example, let's assume, that it's not bad practise to create a visualization, that highly relies on the specific data in one index. What is "the result"? The index linked to the visualization when creating it should be used. I would also like to integrate now with some of the dashboard features:
- I also would like, that e.g. the set filters on the current dashboards are used for this query to ES and would filter the possible tweets shown in the visualization.
- If the user enters a search query this should filter the possible shown tweets.
- If the auto-refresh on the dashboard is set, I would like to do this query again (and determine new possible tweets) in that interval.
- I would also like to get only tweets from the selected time period.
In Kibana there is a
courier service that manages and batches ES calls together. It would be great if I could somehow use this to make regular ES queries, but still have the integration mentioned above. Also sometimes it would be useful to make custom ES queries, but only apply some of the above mentioned filters.
Another example from more real-world scenario. We have a monitoring list, that works on logs sent by devices into ES. We determine all possible devices (terms aggregation over the device id field) and afterwards check for every device the latest log message, to see how long it haven't had logged to ES. It would be great to be possible to integrate that as a Kibana plugin, but we need to do several "chained" ES calls, that still would be great to be integrated with auto refresh and possibly the set query/filters. But these requests for example should not take the time range into account, since the job is to determine the last time a device logged.
Are there any (clean) possibilities for this in Kibana yet? Are there any plans or ideas how to implement such methods (e.g. would it be better to create a Kibana server API, and the server queries ES)?
Discuss now ...