Usability question

Elastic Stack Kibana
1

Hi,

we are using the classic ELK stack, mostly with Log4J messages. We have an average of about 200 log entries/second. We have been using syslog-ng with log rotation and still have that around. I appreciate the faceting, statistics and query capabilities of Kibana. However, there is one major use case for us for which I cannot find a good solution in Kibana, and that is:

  • search for a message indicating or identifying a particular problem
  • "look around", i.e. before and after, that message to see root causes and other things going on for that process

Now with a traditional log file + less this is very easy, using the search function in less I can get to the line I need and then I can just scroll up and down to see other messages at that time. If I searched for example for a session ID, this ID will be highlighted in less which makes it easy to see what happened in the session, but also any concurrent processes that might have interferred with anything.

With Kibana, I can find the message I look for very efficiently, but then "zooming out", i.e. checking what happened before and after that is very cumbersome, so I could adjust the time range to around that line and then remove the search query, but then how would I find and highlight the stuff that's interesting for me?

Is there any functionality that allows me to "scroll" through results while highlighting certain terms? Or can this "find message, see what was going on at that time" use case be done in Kibana?

Thanks,

Stefan

2

I am not aware of any easy way to do it at the moment. It is however a very popular request, and there is an open issue available on GitHub for this type of functionality.

3

I see. Someone there commented "Kibana is essentially a novelty without this." and I agree. Yes, graphing is easier and searching is more powerful. But in our case this comes at the expense of

  • RAM: several GB vs. <20 MB
  • CPU usage: ~3 VM-Ghz (80% of a 4GHz CPU) vs 15 VM-MHz (1.5% of a 1GHz VM)
  • disk space: ~10GB/day vs. <100MB/day for xz-compressed files
  • speed: Kibana and Elasticsearch are slower to retrieve and display data than less, except for the current, uncompressed log file because the old logging VM has slow storage

and now even usabililty because of this. It's a bit sad because in principle it is a very nice tool. But a 300-fold RAM usage, a 200-fold CPU usage and a 100-fold storage usage are quite hard to justify.

Stefan