Nginx Kibana: Bad Gateway 502

Elastic Stack Kibana
1

Hello, I use Nginx as a web server / proxy. I created the following nginx *.conf file for Kibana.

root@dsme01:~# cat /etc/nginx/sites-available/kibana.conf
server {
    # Update this line to be your domain
    server_name kibana.intern.example.com;

    # These shouldn't need to be changed
    listen kibana.intern.example.com:80;
    return 301 https://$host$request_uri;
}


server {
    server_name kibana.intern.example.com;

    listen kibana.intern.example.com:443 ssl;

    ssl_certificate /etc/ssl/certs/kibana.intern.example.com.crt;
    ssl_certificate_key /etc/ssl/private/kibana.intern.example.com.key;
    ssl_dhparam /etc/ssl/certs/dhparams.pem;
    
    location / {

        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
        proxy_pass https://192.168.150.20:5601;
        proxy_redirect off;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $http_host;
        proxy_cache_bypass $http_upgrade;
        }

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

}

When calling up the FQDN via the browser, I get the above error message and the following log entry

2022/12/19 13:19:02 [error] 2740#2740: *101 upstream sent too big header while reading response header from upstream, client: 192.168.30.62, server: kibana.intern.example.com, request: "GET /kibana/login?next=%2Fkibana%2Fkibana%2Flogin%3Fnext%3D%252Fkibana%252Fkibana%252Flogin%253Fnext%253D%25252Fkibana%25252Fkibana%25252Flogin%25253Fnext%25253D%2525252Fkibana%2525252Fkibana%2525252Flogin%2525253Fnext%2525253D%252525252Fkibana%252525252Fkibana%252525252Flogin%252525253Fnext%252525253D%25252525252Fkibana%25252525252Fkibana%25252525252Flogin%25252525253Fnext%25252525253D%2525252525252Fkibana%2525252525252Fkibana%2525252525252Flogin%2525252525253Fnext%2525252525253D%252525252525252Fkibana%252525252525252Fkibana%252525252525252Flogin%252525252525253Fnext%252525252525253D%25252525252525252Fkibana%25252525252525252Fkibana%25252525252525252Flogin%25252525252525253Fnext%25252525252525253D%2525252525252525252Fkibana%2525252525252525252Fkibana%2525252525252525252Flogin%2525252525252525253Fnext%2525252525252525253D%252525252525252525252Fkibana%252525252525252525252Fkibana%252525252525252525252Flogin%252525252525252525253Fnext%252525252525252525253D%25252525252525252525252Fkibana%25252525252525252525252Fkibana%25252525252525252525252Flogin%25252525252525252525253Fnext%25252525252525252525253D%2525252525252525252525252Fkibana%2525252525252525252525252Fkibana%2525252525252525252525252Flogin%2525252525252525252525253Fnext%2525252525252525252525253D%252525252525252525252525252Fkibana%252525252525252525252525252Fkibana%252525252525252525252525252Flogin%252525252525252525252525253Fnext%252525252525252525252525253D%25252525252525252525252525252Fkibana%25252525252525252525252525252Fkibana%25252525252525252525252525252Flogin%25252525252525252525252525253Fnext%25252525252525252525252525253D%2525252525252525252525252525252Fkibana%2525252525252525252525252525252Fkibana%2525252525252525252525252525252Flogin%2525252525252525252525252525253Fnext%2525252525252525252525252525253D%252525252525252525252525252525252Fkibana%2525252525252525252525

Kibana I'm running

root@dsme01:~# netstat -tulpen | grep 5601
tcp        0      0 192.168.150.20:5601     0.0.0.0:*               LISTEN      138        88646      1744/node

What am I doing wrong?

Greetings from Stefan Harbich

2

@azasypkin can we please get some help here? Thanks!

3

In addition, my kibana.yml configuration

server.port: 5601
server.host: "kibana.intern.example.com" # IP 192.168.150.20
server.basePath: "/kibana"
server.name: "Kibana-Server-Harbich"
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/config/certs/kibana.intern.example.com.crt
server.ssl.key: /etc/kibana/config/certs/kibana.intern.example.com.key
elasticsearch.hosts: ["https://elasticsearch.intern.example.com:9200"]
elasticsearch.username: "kibana"
elasticsearch.password: "######"
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/config/certs/HarbichCA.cacert.pem" ]
logging.appenders.file.type: file
logging.appenders.file.fileName: /var/log/kibana/kibana.log
logging.appenders.file.layout.type: json
logging.root.appenders: [default, file]
pid.file: /run/kibana/kibana.pid
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://192.168.20.10:9200'], ca_trusted_fingerprint: b973d7025b9cffae3c40d4......}]
xpack.security.encryptionKey: "###....
4

If I read your nginx config correctly, you serve Kibana from the root path (/), and not from /kibana sub-path. If you plan to continue serving Kibana from the root path, then you need to remove this setting from kibana.yml (and once you do this, try to open Kibana in a private browser tab to make sure there is no stale cookie in play).

Best,
Oleg

5

You are the best Oleg. It works. A heartfelt thank you. I wish you a Merry Christmas.

1 Like
6

Thanks for reporting back, glad it helped!

Happy holidays @sharbich!

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.