Kibana service fails to start

UPDATE: SOLVED!!!

It appears that when you set a log file destination in the kibana configuration file you cannot just say...

logging.dest: "/home/username/kibanalog.txt"

If you do this while tailing systemctl output via sudo journalctl -fu kibana.service you will see that the service fails to create and open this file. Here is what the error looks like while tailing journalctl output...

Apr 19 20:16:45 a-s6bac55wo88z systemd[1]: Started Kibana.
Apr 19 20:16:45 a-s6bac55wo88z systemd[1]: Starting Kibana...
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: events.js:174
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: throw er; // Unhandled 'error' event
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: ^
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: Error: EACCES: permission denied, open '/home/syost/kibanalog.txt'
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: Emitted 'error' event at:
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: at errorOrDestroy (internal/streams/destroy.js:107:12)
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: at WriteStream.onerror (_stream_readable.js:734:7)
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: at WriteStream.emit (events.js:198:13)
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: at lazyFs.open (internal/fs/streams.js:277:12)
Apr 19 20:16:50 a-s6bac55wo88z kibana[6684]: at FSReqWrap.args [as oncomplete] (fs.js:140:20)
Apr 19 20:16:50 a-s6bac55wo88z systemd[1]: kibana.service: main process exited, code=exited, status=1/FAILURE
Apr 19 20:16:50 a-s6bac55wo88z systemd[1]: Unit kibana.service entered failed state.
Apr 19 20:16:50 a-s6bac55wo88z systemd[1]: kibana.service failed.
Apr 19 20:16:53 a-s6bac55wo88z systemd[1]: kibana.service holdoff time over, scheduling restart.

The solution I took was to create a file in /usr/share/kibana called kibanalog.txt and then change the ownership (both USER and GROUP) of that file.

[/usr/share/kibana] > sudo touch kibanalog.txt
[/usr/share/kibana] > sudo chown kibana:kibana kibanalog.txt

After doing this I updated my kibana configuration file to reflect this new change as such...

logging.dest: "/usr/share/kibana/kibanalog.txt"

Now when restarting the kibana service via sudo systemctl restart kibana it starts and runs perfectly. Also I verified that kibana is listening on port 5601 to make sure there were no port conflicts via netstat -plntu and it was...

[/usr/share/kibana] > netstat -plntu
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:5601          0.0.0.0:*               LISTEN      -                   
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      -                   

I'm not sure why kibana is acting as it's own user. If someone could enlighten me on why Kibana does this that would be helpful. This creates headaches with permissions.

2 Likes