Hi Team,
I'm Brand new to ELK stack. I'm trying to configure ELK on my RHEL7.5 system. After Good struggle i was able to configure Elasticsearch and it was up and running .
FYI
Now i've configured Kibana and here is kibana.yml
# For more configuration options see the configuration guide for Kibana in
# https://www.elastic.co/guide/index.html
# =================== System: Kibana Server ===================
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "10.66.36.44"
# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""
# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# Defaults to `false`.
#server.rewriteBasePath: false
# Specifies the public URL at which Kibana is available for end users. If
# `server.basePath` is configured this URL should end with the same basePath.
#server.publicBaseUrl: "http://10.66.36.44"
# The maximum payload size in bytes for incoming server requests.
#server.maxPayload: 1048576
# The Kibana server's name. This is used for display purposes.
server.name: "kibana-monitoring"
# =================== System: Kibana Server (Optional) ===================
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key
# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://10.66.36.44:9200"]
# If your Elasticsearch is protected with basic authentication, these settings provide
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
elasticsearch.username: "kibana_system"
elasticsearch.password: "66CXz0vg4oS+2WxPihGP"
# Kibana can also authenticate to Elasticsearch via "service account tokens".
# Service account tokens are Bearer style tokens that replace the traditional username/password based configuration.
# Use this token instead of a username/password.
# elasticsearch.serviceAccountToken: "my_token"
# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500
# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000
# The maximum number of sockets that can be used for communications with elasticsearch.
# Defaults to `Infinity`.
#elasticsearch.maxSockets: 1024
# Specifies whether Kibana should use compression for communications with elasticsearch
# Defaults to `false`.
#elasticsearch.compression: false
# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]
# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}
# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000
# =================== System: Elasticsearch (Optional) ===================
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key
# Enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/http_ca.crt" ]
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
elasticsearch.ssl.verificationMode: none
# =================== System: Logging ===================
# Set the value of this setting to off to suppress all logging output, or to debug to log everything. Defaults to 'info'
#logging.root.level: debug
# Enables you to specify a file where Kibana stores log output.
logging:
appenders:
file:
type: file
fileName: /var/log/kibana/kibana.log
layout:
type: json
root:
appenders:
- default
- file
# layout:
# type: json
# Logs queries sent to Elasticsearch.
#logging.loggers:
# - name: elasticsearch.query
# level: debug
# Logs http responses.
#logging.loggers:
# - name: http.server.response
# level: debug
# Logs system usage information.
#logging.loggers:
# - name: metrics.ops
# level: debug
# =================== System: Other ===================
# The path where Kibana stores persistent data not saved in Elasticsearch. Defaults to data
#path.data: data
# Specifies the path where Kibana creates the process ID file.
pid.file: /run/kibana/kibana.pid
Logs are showing like below
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:22:13.629+01:00","message":"Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization","log":{"level":"INFO","logger":"plugins.ml"},"process":{"pid":20718},"trace":{"id":"fc09ec4a4a6431a8a9ee691e61e6f6fe"},"transaction":{"id":"308bdcba7a1a8e1e"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:22:13.691+01:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":20718},"trace":{"id":"9090fdc9998d77f00361d5d3d7c27cba"},"transaction":{"id":"232b21bed05f5e8d"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:26:22.798+01:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"total_all_statuses\":0,\"updating\":0},\"fleet_server\":{\"total_all_statuses\":0,\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"updating\":0,\"num_host_urls\":0}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":20718},"trace":{"id":"d23b454315a17b7c2f968f35a7ac3445"},"transaction":{"id":"7ececbea9be35e38"}}
{"process":{"pid":21904,"path":"/run/kibana/kibana.pid"},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:34:34.592+01:00","message":"pid file already exists at /run/kibana/kibana.pid","log":{"level":"WARN","logger":"environment"},"trace":{"id":"59e4502d810078491b8b839d845577ea"},"transaction":{"id":"6581c785703b09b8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:34:34.635+01:00","message":"Kibana process configured with roles: [background_tasks, ui]","log":{"level":"INFO","logger":"node"},"process":{"pid":21904},"trace":{"id":"59e4502d810078491b8b839d845577ea"},"transaction":{"id":"6581c785703b09b8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:41:22.867+01:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"total_all_statuses\":0,\"updating\":0},\"fleet_server\":{\"total_all_statuses\":0,\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"updating\":0,\"num_host_urls\":0}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":20718},"trace":{"id":"d23b454315a17b7c2f968f35a7ac3445"},"transaction":{"id":"6b7005d4051b610e"}}
Here is Journalctl -u kibana.service log
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.918+01:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.979+01:00][INFO ][plugins.synthetics] Installed synthetics index templates
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.001+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: scheduled with interval 1h
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.455+01:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/screenshotting/chr
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.629+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.691+01:00][INFO ][status] Kibana is now available (was degraded)
Mar 26 14:26:22 cn11575apld001 kibana[20718]: [2023-03-26T14:26:22.798+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":
Mar 26 14:41:22 cn11575apld001 kibana[20718]: [2023-03-26T14:41:22.867+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":
Mar 26 14:56:22 cn11575apld001 kibana[20718]: [2023-03-26T14:56:22.919+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":
Here is the curl status for kibana
[devopsadmin@cn11575apld001 ~]$ sudo curl -XGET http://10.66.36.44:5601/ -I
HTTP/1.1 302 Found
location: /login?next=%2F
x-content-type-options: nosniff
referrer-policy: no-referrer-when-downgrade
content-security-policy: script-src 'self' 'unsafe-eval'; worker-src blob: 'self'; style-src 'unsafe-inline' 'self'
kbn-name: kibana-monitoring
kbn-license-sig: ebbec1d966d0c7e52e8295dbf6b07eb145e9a03b8fe1e3aee542b200ebfc7d3c
cache-control: private, no-cache, no-store, must-revalidate
content-length: 0
Date: Sun, 26 Mar 2023 13:57:55 GMT
Connection: keep-alive
Keep-Alive: timeout=120
Port status :
[devopsadmin@cn11575apld001 ~]$ sudo lsof -i:5601
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
node 20718 kibana 57u IPv4 5955954 0t0 TCP cn11575apld001:esmagent (LISTEN)
Service Status here
[devopsadmin@cn11575apld001 ~]$ sudo systemctl status kibana.service
● kibana.service - Kibana
Loaded: loaded (/usr/lib/systemd/system/kibana.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2023-03-26 14:21:49 BST; 37min ago
Docs: https://www.elastic.co
Main PID: 20718 (node)
CGroup: /system.slice/kibana.service
└─20718 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.906+01:00][INFO ][plugins.fleet] Fleet setup completed
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.918+01:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.979+01:00][INFO ][plugins.synthetics] Installed synthetics index templates
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.001+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: scheduled with interval 1h
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.455+01:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/scr...adless_shell
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.629+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.691+01:00][INFO ][status] Kibana is now available (was degraded)
Mar 26 14:26:22 cn11575apld001 kibana[20718]: [2023-03-26T14:26:22.798+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealth...
Mar 26 14:41:22 cn11575apld001 kibana[20718]: [2023-03-26T14:41:22.867+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealth...
Mar 26 14:56:22 cn11575apld001 kibana[20718]: [2023-03-26T14:56:22.919+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealth...
Hint: Some lines were ellipsized, use -l to show in full.
Still i'm unable to bring up kibana in browser. Kinldy help me on this .
Thanks in Advance.
Thanks,
Siva
