Can not get kibana logs

[informatico]

I am unable to get kibana logs.

/var/log/kibana/kibana.stdout as well as kibana.stderr seem to have no logs being written to them (I am getting undefined errors in my Kibana)

In the kibana.yml file, I have then also set
logging.dest: /var/logs/kibana.log
while inserting a kibana.log file into this directory... but again nothing is being written to the file.

How do I obtain the Kibana logs?

Thanks in advance.

[weltenwort (Felix Stürmer)]

Hi @informatico,

the default log file location depends on the package. Which package format did you use to install Kibana (deb, rpm, tgz)?

[informatico]

I believe the RPMs were downloaded directly, so where would they be in this case?

Thanks

[weltenwort (Felix Stürmer)]

The default log output destination depends on the init system your linux distribution uses:

For SysV stdout and stderr of Kibana would be written to /var/log/kibana.{stdout,stderr}. Changing logging.dest to something besides stdout will cause these files to be empty. One thing to keep in mind is that the Kibana process needs to have write access to these files or the file configured in logging.dest. The default init script should take care of that.

For systemd the output is captured by journald and can be inspected using journalctl.

[informatico]

I figured it must be permissions as it was -rw-r--r-- for both files.

So I performed chmod ugoa+rwx on both files, to give all users all kinds of permissions, just to see what errors are being written.... however still nothing is being written to the file

logging.dest is stdout in the kibana.yml file.

Any ideas?

[weltenwort (Felix Stürmer)]

How are you starting Kibana? Via SysV Init, systemd or a custom way?

[informatico]

Kibana is running in a Centos 7 VM.

systemd was used to start Kibana via systemctl command

[weltenwort (Felix Stürmer)]

Under systemd with logging.dest on default or set to stdout you should be able to see the Kibana output using journalctl -u kibana.service (assuming the Kibana unit file is called kibana.service).

[informatico]

When I invoke that I get :
No journal files were found.
-- No entries --
kibana.service is located in ./etc/systemd/system/

I guess this is the problem? I have restarted systemd-jounald, but have had no luck.
Am I supposed to create this file manually? If so.. how and where?

Sorry this thread is going on a little, I'm a really nooby. Ultimately I am trying to get to the bottom of a 404 error I am getting with the Kibana reporting plugin (thread: https://discuss.elastic.co/t/reporting-error-404-and-kibana-unable-to-connect/ ).

I really appreciate your help here Felix. Many thanks.

[weltenwort (Felix Stürmer)]

Don't worry, that's what this forum is here for.

I'm not very familiar with CentOS 7, but it seems that it is using journald only for transient logs and still forwards them to rsyslogd. So whether rsyslogd persists the log anywhere depends on your configuration.

To enable persistence of the journald journal a directory called /var/log/journal must be created (according to chapter 20.10.5. of the RHEL7 System Admin Guide).