Sniffing Kibana data behind proxy

[cuneyt]

I have configured a reverse proxy for Kibana using NginX and a simple NodeJs script that restricts Kibana access with user/pass. It works as expected. Kibana is proxied through NodeJS script. But I would also like to sniff this flowing data. But for some reason I cannot read the HTML that is generated when I proxy Kibana. If I proxy a simple website I can see the code. I use the same code for both.

This is what I see when I proxy a simple web page:

But when I proxy the Kibana I see a very weird encoding:

What I expect to see is some HTML code that's being sent to the client/browser, Is this an encoding issue? What kind of encoding does Kibana use? Or is it because the data is encrypted? Is it possible to read this?

[Brandon_Kobel]

@cuneyt do you have Kibana itself running over http or https? If you try to access Kibana directly without going through your proxy, you should be able to determine whether it's http or https.

[cuneyt]

Hi Brandon. I do the direct access using an address like http://54.89.89.xxx:5601 So it seems to be http. The thing I normally close port 5601 to direct access for the production environment. So I make sure that requests only go thru an authentication app.

The code is pretty simple. 5601 port is closed to access from outside. All kibana routes are directed to this nodejs app. It uses the http-proxy and a middleware function that sniffs the code. If I change "localhost:5601" to anything like "google.com", "yahoo.com" I can actually see the generated HTML code of that websites. But for kibana website all I see is this weird encoding. So I think it's a Kibana question rather than a NodeJs question.

var transformerFunction = function (data, req, res) {
  console.log(data.toString('ascii'));
  return data;
};

var apiProxy = require('http-proxy').createProxyServer();

app.all("/ui/|/api/|/es_admin/|/elasticsearch/|/app/|/bundles/|/kibana|/kibana5|/status|/plugins", require('transformer-proxy')(transformerFunction), function(req, res) {
          apiProxy.web(req, res, { target: 'http://localhost:5601' });
    });

[Brandon_Kobel]

@cuneyt You probably want to be inspecting the response headers, as the Content-Type is text/html; charset=UTF-8 so your .toString('ascii') isn't going to work.

[cuneyt]

Hi Thanks Brandon but I tried all possible encodings including all below but still the same

ascii
base64
binary
hex
ucs2/ucs-2/utf16le/utf-16le
utf8/utf-8

[Brandon_Kobel]

@cuneyt if the browser supports it, we're also using gzip to compress the data, it can be decompressed using something similar to the following

zlib.gunzip(data, function (err, result) {
    console.log(result.toString('utf8'));
});

The response headers should be used to determine whether it's gzipped and the encoding, as this can vary based on the request headers that the browser sets.

[cuneyt]

Thanks Brandon. Decompression! That is the solution. It works! Thanks many times.