Kibana 4.x very slow with large document

Elastic Stack Kibana
1

Hi,

our team recently switched from ES 1.7 / Kibana 3.5 to ES 2.3 / Kibana 4.5 and we experience a very poor performance of Kibana 4.5 when at least one document in the results is "large".

Let me explain: the query returns only 6 documents, and according to the statistics the runtime was OK:

Query time: 1029ms
Response time: 8514ms

During this time the browser hangs for 1-2 sec. (you can see it on the 3 "bubbles" in the top right) which would still be acceptable.

BUT! Opening (clicking the arrow in "Discover") one of the 6 documents (let's call it "_id:doc1") takes well over 10-20 seconds and Firefox throws an "unresponsive script" popup:

_id:doc1 has 21 fields and the "message" field is 4.8MB (really looong text).

The same query with the same results takes only a few short seconds and opening _id:doc1 happens instantly in Kibana 3.

We tried this with Kibana 4.5.1 under Linux Mint with Firefox 47 and also under Windows 7 with Firefox and Chrome. Chrome usually performs even worse than Firefox.

Does anybody else experience the same problem?

2

This is a known issue, and is caused by the time it takes the browser to parse the large JSON response that comes back from Elasticsearch. It's mentioned on this issue, but more in regards to the UX:

There is also this issue which proposes allowing users to hide the contents of known large fields:

There are actually 2 problems here. The slowness of the JSON parsing is further complicated by the slowness of the table rendering in Discover, as mentioned here (and this is likely what is causing the issue for you):

We have talked about using a streaming JSON parser to deal with large response bodies, but that is still in more of the consideration phase. I thought we had an issue open for dealing with large documents specifically, which I thought had some discussion about a streaming parser, but I can't seem to find it.

As for fixing the table rendering, that last issue is probably the one to follow and leave comments on.

3

Hi!

Thank you very much for the links. Unfortunately none of these describes the problem we have.

  1. "Show progress bar for big result sets": like I wrote, the result set is not that big. 6 documents, one with 4.8MB in size, the others a few kilobytes maybe. The whole result set isn't more than 6MB. On a local GBit lan this shouldn't be a problem.

  2. "Loading huge documents": this is more like what I described, but the proposed solution (eliminating huge fields) is a no-go. The huge field is the field we need the most from the result set.

  3. "Slow doc table rendering with large number of fields": this seems more like it, but I don't think that 20-21 fields are that much. When I get the time I'll make a index with only 1 doc, 1 field and make the one field really big, then I'll test if the same problem occurs.

I really would like to make a video, but the information in the docs sensitive, so I'm not allowed to.

Also, I thought about making a timeline graph (similar to the ones shown in the first and third link), but I can't even open the page in Chrome (copied the link from FF). I always get a popup, where I can "Kill" or "Wait". I have clicked "Wait" 5 times in a row, but the page still didn't load. :frowning: