Can kibana create charts similar to afterglow

Elastic Stack Kibana
1

Dear all guru,

Being new to kibana, I wonder if kibana can generate visualization similar to what afterglow is doing for network traffic analysis in Davix Live CD.

My use case is doing network analysis over my team machines (around 10 hosts) against other servers/hosts of my company as well as the internet sites. Geolocation is not my concern as I expect seeing majority traffic either within the team or with my company servers (which are geographically the same location). Afterglow serves my purpose by producing the network group showing clustering of the traffic (ref. afterglow.sourceforge.net). I wish to use the filter function of Kibana to "take away" some hosts interactively so I can focus at the traffic I interested at. Any idea if this can be done? If so, how to do?

Wilson.

2

We're tracking a Github issue around this requests (see below).

We'd love your input / comments on that issue around your use case, how you'd like to organize your data in Elasticsearch and specify the dependencies, and the interactivity (filtering) ideas you mentioned in your post.

3

Thanks for your response, Tanya.

My use case:

My organization use a class A private network (10.X.X.X) while my team use a subnet (10.7.A.X). I want to do some visualization on the network traffic in my network, i.e. how the hosts in my team “communicate” with others. I expect seeing several clusters of IPs:

  1.   A cluster of hosts with 10.7.A.X  meaning my team is accessing hosts/servers inside our team network.

  2.   Certain dedicated hosts  10.X.X.X  meaning my organization servers (DNS, web, etc) is supporting my team

  3.   Certain dedicated common hosts (e.g. yahoo, google, etc) meaning my team is accessing outside service

In this case, the actual geolocation is not relevant to me. In particular, cluster 1 and 2 will be at the same location which doesn’t mean anything to me. At DAVIX 2014 live CD (http://www.secviz.org/node/89), it includes a demonstration (see https://github.com/secviz/davix/wiki/User-Guide:-AfterGlow) on using tshark (to capture traffic as CSV), afterglow (to transform as GraphViz dot file) and neato (to generate png/gif) to visualize the network traffic into something like this: (http://afterglow.sourceforge.net/) (Note: this IS NOT my network, so don’t see previous 3 points)
[Screenshot]

In general, it well serves my purpose. But the problem is when too many links are overlapping each other, I see nothing at all (like the central cluster). Therefore, if I can use the “filter” function of Kibana to filter out certain IPs, I expect to see something interesting.

Similar approach can be used to do relationship analysis, like email traffic.

My question:

  1.    I want to confirm if the current Kibana can do something like this?  If yes, how to do?

  2.   If no, can current Kibana “call” external program for visualization?   Wishfully, it could be something like this: after using tshark to capture the traffic and using elasticsearch to store, I can use the filter of Kibana to select and filter out some ips, then it  will “pipe” the data to my designated script which both afterglow and neato to generate the png/gif file and present at Kibana dashboard.

  3.   If still no, does this request similar to anything in the Kibana development plan which I may see in the near future (say 1-3months)?

Rgds,
Wilson.

4

No, thus the Github issue I linked to above. I'd love for you to add your use case to it as a comment, so we can track it!

While Kibana does not have a default visualization for this, there is something you can use short-term. We have a new addition to the Elastic family, called Packetbeat. It is a small forwarder that runs on the server and captures information based on traffic analytics, including IP topology. They have a topology map view that you can use for what you are trying to do. Currently the view only works with Kibana 3, but will likely be ported to Kibana 4 in the future. Check it out (in test/dev as it's still in beta), and let us know what you think on the Packetbeat topic in this forum.

We don't have a 1-3 month timeframe to implement a general topomap solution in Kibana.

5

Thanks Tanya. Will take your advice to check on Packetbeat.

Wilson.