KIbana upgraded from 5.3.1 to 5.4 but now unable to browse to it


#1

Hi , upgraded my whole Elastic Stack from 5.3.1 to 5.4 yesterday. ES is good, LS is good, all beats inc x-pack (monitoring only) but KB might have an issue because I can't browse to it.. page can't be displayed.

Server is Ubuntu 16.0.4. and on 5.3.1 everything was working perfectly.

issued netstat | grep 5601 but nothing so probably not listening...

When I issue sudo systemctl status kibana its 'running'

@kib01:/etc/kibana$ sudo systemctl status kibana
● kibana.service - Kibana
   Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2017-05-28 15:56:34 AEST; 4min 36s ago
 Main PID: 2608 (node)
    Tasks: 10
   Memory: 310.5M
      CPU: 37.085s
   CGroup: /system.slice/kibana.service
           └─2608 /usr/share/kibana/bin/../node/bin/node --no-warnings /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml

May 28 15:59:23 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T05:59:23Z","tags":[],"pid":2608,"os":{"load":[1.47216796875,2.0263671875,1.87451171875],"mem":{"total":4143366144,"free":127279104},"uptime":26494},"proc":{"uptime":16
May 28 15:59:24 kib01 kibana[2608]: {"type":"log","@timestamp":"2017-05-28T05:59:24Z","tags":["info","optimize"],"pid":2608,"message":"Optimizing and caching bundles for graph, monitoring, ml, kibana, timelion, login, logout and status_page. 
May 28 15:59:38 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T05:59:38Z","tags":[],"pid":2608,"os":{"load":[1.36669921875,1.9755859375,1.85986328125],"mem":{"total":4143366144,"free":108830720},"uptime":26509},"proc":{"uptime":18
May 28 15:59:43 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T05:59:43Z","tags":[],"pid":2608,"os":{"load":[1.65771484375,2.02587890625,1.876953125],"mem":{"total":4143366144,"free":131092480},"uptime":26514},"proc":{"uptime":187
May 28 16:00:24 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T06:00:22Z","tags":[],"pid":2608,"os":{"load":[2.13916015625,2.1025390625,1.9091796875],"mem":{"total":4143366144,"free":115318784},"uptime":26547},"proc":{"uptime":220
May 28 16:00:24 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T06:00:22Z","tags":[],"pid":2608,"os":{"load":[2.36865234375,2.15087890625,1.92578125],"mem":{"total":4143366144,"free":112357376},"uptime":26553},"proc":{"uptime":226.
May 28 16:00:35 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T06:00:35Z","tags":[],"pid":2608,"os":{"load":[4.02099609375,2.4970703125,2.0390625],"mem":{"total":4143366144,"free":108773376},"uptime":26560},"proc":{"uptime":233.40
May 28 16:00:40 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T06:00:40Z","tags":[],"pid":2608,"os":{"load":[5.978515625,2.9736328125,2.19970703125],"mem":{"total":4143366144,"free":116412416},"uptime":26566},"proc":{"uptime":239.
May 28 16:00:43 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T06:00:43Z","tags":[],"pid":2608,"os":{"load":[6.541015625,3.14013671875,2.2578125],"mem":{"total":4143366144,"free":138539008},"uptime":26572},"proc":{"uptime":245.044
May 28 16:01:06 kib01 kibana[2608]: {"type":"ops","@timestamp":"2017-05-28T06:01:06Z","tags":[],"pid":2608,"os":{"load":[6.17724609375,3.12109375,2.25634765625],"mem":{"total":4143366144,"free":117882880},"uptime":26580},"proc":{"uptime":253.
lines 1-20/20 (END)



tail of my /var/log/syslog file
    May 28 15:56:34 kib01 systemd[1]: Started Kibana.
    May 28 15:58:07 kib01 kibana[2608]: {"type":"log","@timestamp":"2017-05-28T05:58:07Z","tags":["plugins","debug"],"pid":2608,"dir":"/usr/share/kibana/plugins","message":"Scanning `/usr/share/kibana/plugins` for plugins"}
    May 28 15:58:07 kib01 kibana[2608]: {"type":"log","@timestamp":"2017-05-28T05:58:07Z","tags":["plugins","debug"],"pid":2608,"dir":"/usr/share/kibana/src/core_plugins","message":"Scanning `/usr/share/kibana/src/core_plugins` for plugins"}
    May 28 15:59:09 kib01 kibana[2608]: {"type":"log","@timestamp":"2017-05-28T05:59:09Z","tags":["plugins","debug"],"pid":2608,"path":"/usr/share/kibana/plugins/x-pack/index.js","message":"Found plugin at /usr/share/kibana/plugins/x-pack/index.js"}
    May 28 15:59:09 kib01 kibana[2608]: {"type":"log","@timestamp":"2017-05-28T05:59:09Z","tags":["plugins","debug"],"pid":2608,"path":"/usr/share/kibana/src/core_plugins/console/index.js","message":"Found plugin at /usr/share/kibana/src/core_plugins/console/index.js"}

Any pointers where it could give a hint on whats going on?

I have 2 KB Servers with exactly the same (hindsight should have done 1 box at a time... next time).


(Mark Walkom) #2

What's in the actual KB log - /var/log/kibana/?


#3

I don't have that folder/file but when I enabled:

logging.quiet: false
logging.verbose: true

in

/etc/kibana/kibana.yml

my /var/log/syslog log was very quiet.. I removed kibana, reboot, re-installed kibana and x-pack and now its good.. I


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.