Kibana service start failed

Hi Team,

I have standalone server where I downloaded elastic and kibana 7.12.1 in RPM on cent os server
After downloading i have install elastic and kibana from below command
rpm -i elasticseach-7.12.1-x86_64.rpm

After this step 1 made configuration changes to run on IP .

Create data and log directory and gave permission elasticsearch group I strated elasticsearch using below command
systemctl start elasticsearch
here are the logs

[2021-07-13T20:20:02,731][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-watcher]
[2021-07-13T20:20:02,732][INFO ][o.e.p.PluginsService ] [node-1] no plugins loaded
[2021-07-13T20:20:02,765][INFO ][o.e.e.NodeEnvironment ] [node-1] using [1] data paths, mounts [[/home (/dev/mapper/cl-home)]], net usable_space [2.1tb], net total_space [2.1tb], types [xfs]
[2021-07-13T20:20:02,766][INFO ][o.e.e.NodeEnvironment ] [node-1] heap size [14gb], compressed ordinary object pointers [true]
[2021-07-13T20:20:02,828][INFO ][o.e.n.Node ] [node-1] node name [node-1], node ID [LAtDya-BQu284bEnsdxowQ], cluster name [oes-cluster], roles [transform, data_frozen, master, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold, ingest]
[2021-07-13T20:20:06,740][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [node-1] [controller/17017] [Main.cc@117] controller (64 bit): Version 7.12.1 (Build 4172997de5701c) Copyright (c) 2021 Elasticsearch BV
[2021-07-13T20:20:07,199][INFO ][o.e.x.s.a.s.FileRolesStore] [node-1] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2021-07-13T20:20:08,565][INFO ][o.e.t.NettyAllocator ] [node-1] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=8mb}]
[2021-07-13T20:20:08,620][INFO ][o.e.d.DiscoveryModule ] [node-1] using discovery type [zen] and seed hosts providers [settings]
[2021-07-13T20:20:09,023][INFO ][o.e.g.DanglingIndicesState] [node-1] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2021-07-13T20:20:09,414][INFO ][o.e.n.Node ] [node-1] initialized
[2021-07-13T20:20:09,415][INFO ][o.e.n.Node ] [node-1] starting ...
[2021-07-13T20:20:09,426][INFO ][o.e.x.s.c.f.PersistentCache] [node-1] persistent cache index loaded
[2021-07-13T20:20:09,518][INFO ][o.e.t.TransportService ] [node-1] publish_address {172.25.23.189:9300}, bound_addresses {172.25.23.189:9300}
[2021-07-13T20:20:09,853][INFO ][o.e.b.BootstrapChecks ] [node-1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2021-07-13T20:20:09,875][INFO ][o.e.c.c.Coordinator ] [node-1] cluster UUID [wpLXNI_RRh-VvWOrxr7P2A]
[2021-07-13T20:20:09,954][INFO ][o.e.c.s.MasterService ] [node-1] elected-as-master ([1] nodes joined)[{node-1}{LAtDya-BQu284bEnsdxowQ}{Fu2CWL0qQCuTOd6bHRn_Ug}{172.25.23.189}{172.25.23.189:9300}{cdfhilmrstw}{ml.machine_memory=33431883776, xpack.installed=true, transform.node=true, ml.max_open_jobs=20, ml.max_jvm_size=15032385536} elect leader, BECOME_MASTER_TASK, FINISH_ELECTION], term: 6, version: 102, delta: master node changed {previous , current [{node-1}{LAtDya-BQu284bEnsdxowQ}{Fu2CWL0qQCuTOd6bHRn_Ug}{172.25.23.189}{172.25.23.189:9300}{cdfhilmrstw}{ml.machine_memory=33431883776, xpack.installed=true, transform.node=true, ml.max_open_jobs=20, ml.max_jvm_size=15032385536}]}
[2021-07-13T20:20:10,053][INFO ][o.e.c.s.ClusterApplierService] [node-1] master node changed {previous , current [{node-1}{LAtDya-BQu284bEnsdxowQ}{Fu2CWL0qQCuTOd6bHRn_Ug}{172.25.23.189}{172.25.23.189:9300}{cdfhilmrstw}{ml.machine_memory=33431883776, xpack.installed=true, transform.node=true, ml.max_open_jobs=20, ml.max_jvm_size=15032385536}]}, term: 6, version: 102, reason: Publication{term=6, version=102}
[2021-07-13T20:20:10,104][INFO ][o.e.h.AbstractHttpServerTransport] [node-1] publish_address {172.25.23.189:9200}, bound_addresses {172.25.23.189:9200}
[2021-07-13T20:20:10,104][INFO ][o.e.n.Node ] [node-1] started
[2021-07-13T20:20:10,367][INFO ][o.e.l.LicenseService ] [node-1] license [160074e9-883e-4e58-9b77-a916db9c0d80] mode [basic] - valid
[2021-07-13T20:20:10,369][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [node-1] Active license is now [BASIC]; Security is disabled
[2021-07-13T20:20:10,373][INFO ][o.e.g.GatewayService ] [node-1] recovered [7] indices into cluster_state
[2021-07-13T20:20:11,006][INFO ][o.e.c.r.a.AllocationService] [node-1] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.ds-ilm-history-5-2021.07.13-000001][0], [.kibana-event-log-7.12.1-000001][0]]]).
[2021-07-13T20:20:36,110][INFO ][o.e.t.LoggingTaskListener] [node-1] 62 finished with response BulkByScrollResponse[took=222.1ms,timed_out=false,sliceId=null,updated=9,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=,search_failures=]
[2021-07-13T20:20:36,156][INFO ][o.e.t.LoggingTaskListener] [node-1] 69 finished with response BulkByScrollResponse[took=184.3ms,timed_out=false,sliceId=null,updated=11,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=,search_failures=]
[2021-07-13T20:31:15,681][INFO ][o.e.t.LoggingTaskListener] [node-1] 4646 finished with response BulkByScrollResponse[took=26.6ms,timed_out=false,sliceId=null,updated=9,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=,search_failures=]
[2021-07-13T20:31:15,763][INFO ][o.e.t.LoggingTaskListener] [node-1] 4642 finished with response BulkByScrollResponse[took=117.9ms,timed_out=false,sliceId=null,updated=11,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=,search_failures=]
[2021-07-13T20:42:55,889][INFO ][o.e.t.LoggingTaskListener] [node-1] 5783 finished with response BulkByScrollResponse[took=25.6ms,timed_out=false,sliceId=null,updated=9,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=,search_failures=]
[2021-07-13T20:42:55,970][INFO ][o.e.t.LoggingTaskListener] [node-1] 5779 finished with response BulkByScrollResponse[took=116.2ms,timed_out=false,sliceId=null,updated=11,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=,search_failures=]

Then I have started kibana using
systemctl start kibana
here is the log of kibana I'm not aable to understand the error and where I'm stuck
Jul 13 20:20:34 localhost kibana: An unexpected error occurred in the logging pipeline: Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'
Jul 13 20:20:34 localhost kibana: An unexpected error occurred in the logging pipeline: Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'
Jul 13 20:20:34 localhost kibana: An unexpected error occurred in the logging pipeline: Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'
Jul 13 20:20:34 localhost kibana: An unexpected error occurred in the logging pipeline: Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'
Jul 13 20:20:34 localhost kibana: An unexpected error occurred in the logging pipeline: Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'
Jul 13 20:20:34 localhost kibana: There was a problem (Error: EACCES: permission denied, open '/var/log/kibana/kibana.log') in logReporter and it has been destroyed.
Jul 13 20:20:34 localhost kibana: [Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'] {
Jul 13 20:20:34 localhost kibana: errno: -13,
Jul 13 20:20:34 localhost kibana: code: 'EACCES',
Jul 13 20:20:34 localhost kibana: syscall: 'open',
Jul 13 20:20:34 localhost kibana: path: '/var/log/kibana/kibana.log'
Jul 13 20:20:34 localhost kibana: }
Jul 13 20:20:38 localhost kibana: An unexpected error occurred in the logging pipeline: Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'
Jul 13 20:20:38 localhost kibana: There was a problem (Error: EACCES: permission denied, open '/var/log/kibana/kibana.log') in logReporter and it has been destroyed.
Jul 13 20:20:38 localhost kibana: [Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'] {
Jul 13 20:20:38 localhost kibana: errno: -13,
Jul 13 20:20:38 localhost kibana: code: 'EACCES',
Jul 13 20:20:38 localhost kibana: syscall: 'open',
Jul 13 20:20:38 localhost kibana: path: '/var/log/kibana/kibana.log'
Jul 13 20:20:38 localhost kibana: }
Jul 13 20:20:38 localhost kibana: An unexpected error occurred in the logging pipeline: Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'
Jul 13 20:20:38 localhost kibana: There was a problem (Error: EACCES: permission denied, open '/var/log/kibana/kibana.log') in logReporter and it has been destroyed.
Jul 13 20:20:38 localhost kibana: [Error: EACCES: permission denied, open '/var/log/kibana/kibana.log'] {
Jul 13 20:20:38 localhost kibana: errno: -13,
Jul 13 20:20:38 localhost kibana: code: 'EACCES',
Jul 13 20:20:38 localhost kibana: syscall: 'open',
Jul 13 20:20:38 localhost kibana: path: '/var/log/kibana/kibana.log'
Jul 13 20:20:38 localhost kibana: }

And I have given permission of kibana group to /var/log/kibana/kibana.log
image

Kindly help.

Thank you in advance.

Swarali Vartak.

Hi @swarali_vartak ,

It looks like you faced the same problem described here

Can you check the provided solution and update us?

Regards, Dima

Hi,

Thank you for your reponse.
Issue Resolved.

I have restarted the elasticsearch and kibana after giving permission of kibana group and after some time kibana service started.
Other issue was kibana UI was not loading that is because of Firefox(browser issue), when I tried in google chrome kibana was up and running.

Regards,

Swarali Vartak

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.