Kibana not compressing HTTP responses

Elastic Stack Kibana
1

Recently I've updated to Kibana 7.11.1 and I've noticed slow performance specially in Discover when many documents are returned.

After some investigation I've noticed that Kibana is no longer compressing the response of queries. My browser is providing Accept-Encoding: gzip, deflate but Kibana seems to be ignoring that. The Content-Encoding header is missing:

image
image883×789 77.1 KB

Interestingly, this is not the behaviour Kibana has with other types of documents like status:

image
image882×792 121 KB

I've forced compression in the configuration (although the default is true)

server.compression.enabled: true

but the situation is the same.

Any ideas?

Thanks!
Juan

2

Hi, it looks like you may have found a bug. Can you please file an issue in our tracker at https://github.com/elastic/kibana/issues/new

As another demonstration, can you try using curl -i to send a request directly to the Kibana server (make sure the request is not hitting a proxy) and show the response headers?

3

Thanks for your answer.

I don't know how to construct a full /internal/bsearch request using curl (Kibana expects session headers...)

But I'll give it a try.

4

OK. I think I have it. I've used -v that shows all the headers in both directions to be sure.

I ask for gzipped content (/internal/bsearch) but it does not come back that way:

# curl -s -v -u elastic:elastic -XPOST 'localhost:5601/internal/bsearch' -H 'Content-Type: application/json' -H 'Accept-Encoding: gzip, deflate' -H 'kbn-xsrf: kibana' -d '{"batch":[{"request":{"params":{"index":"logs*:*","body":{"version":true,"size":500,"sort":[{"@timestamp":{"order":"desc","numeric_type":"date","unmapped_type":"boolean"}}],"aggs":{"2":{"date_histogram":{"field":"@timestamp","fixed_interval":"30s","time_zone":"Europe/Dublin","min_doc_count":1}}},"fields":[],"script_fields":{},"stored_fields":["*"],"_source":{"excludes":[]},"query":{"bool":{"must":[],"filter":[{"match_all":{}},{"range":{"@timestamp":{"gte":"2021-03-18T21:46:39.629Z","lte":"2021-03-18T22:01:39.629Z","format":"strict_date_optional_time"}}}],"should":[],"must_not":[]}},"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}},"fragment_size":2147483647}},"preference":1616104872586}},"options":{"strategy":"ese","isStored":false,"isRestore":false}}]}'

* About to connect() to localhost port 5601 (#0)
*   Trying ::1...
* Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5601 (#0)
* Server auth using Basic with user 'elastic'
> POST /internal/bsearch HTTP/1.1
> Authorization: Basic xxxxxxxxx
> User-Agent: curl/7.29.0
> Host: localhost:5601
> Accept: */*
> Content-Type: application/json
> Accept-Encoding: gzip, deflate
> kbn-xsrf: kibana
> Content-Length: 830
>
* upload completely sent off: 830 out of 830 bytes
< HTTP/1.1 200 OK
< content-type: application/x-ndjson
< connection: keep-alive
< transfer-encoding: chunked
< kbn-name: xxxxxxxxx
< kbn-license-sig: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
< cache-control: private, no-cache, no-store, must-revalidate
< Date: Thu, 18 Mar 2021 22:26:59 GMT
<
{"id":0,"result":{"rawResponse":{"took":133,"timed_out":false,"_shards":{"total":394,"successful":394,"skipped":344,"failed":0},"_clusters":{"total":1,"successful":1,"skipped":0},"hits":{"total":1713499,"max_score":null,"hits":[{...
...
...
...
(10 KBytes of documents)

On the other hand, if I ask for a different type of request (like Dev Tools) it comes compressed no problem:

# curl -s -v -u elastic:elastic -XPOST 'localhost:5601/api/console/proxy?path=_mapping&method=GET' -H 'Content-Type: application/json' -H 'Accept-Encoding: gzip, deflate' -H 'kbn-xsrf: kibana'

* About to connect() to localhost port 5601 (#0)
*   Trying ::1...
* Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5601 (#0)
* Server auth using Basic with user 'elastic'
> POST /api/console/proxy?path=_mapping&method=GET HTTP/1.1
> Authorization: Basic xxxxxxxx
> User-Agent: curl/7.29.0
> Host: localhost:5601
> Accept: */*
> Content-Type: application/json
> Accept-Encoding: gzip, deflate
> kbn-xsrf: kibana
>
< HTTP/1.1 200 OK
< warning: 299 Elasticsearch-7.11.1-ff17057114c2199c9c1bbecc727003a907c0db7a "this request accesses system indices: [.async-search, .kibana_task_manager_1, .security-7, .tasks], but in a future major version, direct access to system indices will be prevented by default"
< content-type: application/json; charset=UTF-8
< kbn-name: xxxxxxxx
< kbn-license-sig: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
< cache-control: private, no-cache, no-store, must-revalidate
< vary: accept-encoding
< content-encoding: gzip
< Date: Thu, 18 Mar 2021 22:42:55 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
< Transfer-Encoding: chunked
<
?e??l?6?I?Q??lR[??$!f?????Z???u?????ͧ/w߫??s'Ȝp????͗?(?VΦ.'H0??U?ӆ]???˯5Ő?vE?uվ?/X?o??z??4Np?8?Wr?_? ?.F?q???c???}?????o??5??+???(+?8u?i\$:?>R_?Xd?с??0&?m?? ??<?#SAY\????#?||O??@a?[???$??͙?Q
>?[?b?i??f?|?F????cu?"?8???f?G]&SY<?A????0?0??>?_?S?'=DK?MI???(N1ي?E??q?Y?i??B???Y?ؘ ????'?&???$ɂ/dۭ??m?7??R??e?hNeS?dd?Sp??m?ژ?????k??Ѳ1??zs|?ז,?0????ӕ?Ѻa$??.?D(?9ӯ.??&GYt?<?K?Vq??K?s3?V"??8
...
...
...

Case open: Kibana not compressing some HTTP responses · Issue #94998 · elastic/kibana · GitHub

Thanks!

5

Workaround: In the interim I'm moving the compression layer to Apache.

If you wish to do the same:

#1
Install mod_deflate and in /etc/httpd/conf/httpd.conf add

LoadModule deflate_module modules/mod_deflate.so

#2
In your virtual host add (Typical file name:/etc/httpd/conf.d/kibana.conf)

SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI ".(?:gif|jpe?g|png)$" no-gzip

#3
And disable compression in Kibana. In /etc/kibana/kibana.yml add

server.compression.enabled: false

I'm getting satisfactory results.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.