Kibana8.6.2 not loading in browser

Hi Team,

I'm Brand new to ELK stack. I'm trying to configure ELK on my RHEL7.5 system. After Good struggle i was able to configure Elasticsearch and it was up and running .

FYI
image

Now i've configured Kibana and here is kibana.yml

# For more configuration options see the configuration guide for Kibana in
# https://www.elastic.co/guide/index.html

# =================== System: Kibana Server ===================
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "10.66.36.44"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# Defaults to `false`.
#server.rewriteBasePath: false

# Specifies the public URL at which Kibana is available for end users. If
# `server.basePath` is configured this URL should end with the same basePath.
#server.publicBaseUrl: "http://10.66.36.44"

# The maximum payload size in bytes for incoming server requests.
#server.maxPayload: 1048576

# The Kibana server's name. This is used for display purposes.
server.name: "kibana-monitoring"

# =================== System: Kibana Server (Optional) ===================
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://10.66.36.44:9200"]

# If your Elasticsearch is protected with basic authentication, these settings provide
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
elasticsearch.username: "kibana_system"
elasticsearch.password: "66CXz0vg4oS+2WxPihGP"

# Kibana can also authenticate to Elasticsearch via "service account tokens".
# Service account tokens are Bearer style tokens that replace the traditional username/password based configuration.
# Use this token instead of a username/password.
# elasticsearch.serviceAccountToken: "my_token"

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# The maximum number of sockets that can be used for communications with elasticsearch.
# Defaults to `Infinity`.
#elasticsearch.maxSockets: 1024

# Specifies whether Kibana should use compression for communications with elasticsearch
# Defaults to `false`.
#elasticsearch.compression: false

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# =================== System: Elasticsearch (Optional) ===================
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/http_ca.crt" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
elasticsearch.ssl.verificationMode: none

# =================== System: Logging ===================
# Set the value of this setting to off to suppress all logging output, or to debug to log everything. Defaults to 'info'
#logging.root.level: debug

# Enables you to specify a file where Kibana stores log output.
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
#  layout:
#    type: json
# Logs queries sent to Elasticsearch.
#logging.loggers:
#  - name: elasticsearch.query
#    level: debug

# Logs http responses.
#logging.loggers:
#  - name: http.server.response
#    level: debug

# Logs system usage information.
#logging.loggers:
#  - name: metrics.ops
#    level: debug

# =================== System: Other ===================
# The path where Kibana stores persistent data not saved in Elasticsearch. Defaults to data
#path.data: data

# Specifies the path where Kibana creates the process ID file.
pid.file: /run/kibana/kibana.pid

Logs are showing like below

{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:22:13.629+01:00","message":"Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization","log":{"level":"INFO","logger":"plugins.ml"},"process":{"pid":20718},"trace":{"id":"fc09ec4a4a6431a8a9ee691e61e6f6fe"},"transaction":{"id":"308bdcba7a1a8e1e"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:22:13.691+01:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":20718},"trace":{"id":"9090fdc9998d77f00361d5d3d7c27cba"},"transaction":{"id":"232b21bed05f5e8d"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:26:22.798+01:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"total_all_statuses\":0,\"updating\":0},\"fleet_server\":{\"total_all_statuses\":0,\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"updating\":0,\"num_host_urls\":0}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":20718},"trace":{"id":"d23b454315a17b7c2f968f35a7ac3445"},"transaction":{"id":"7ececbea9be35e38"}}
{"process":{"pid":21904,"path":"/run/kibana/kibana.pid"},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:34:34.592+01:00","message":"pid file already exists at /run/kibana/kibana.pid","log":{"level":"WARN","logger":"environment"},"trace":{"id":"59e4502d810078491b8b839d845577ea"},"transaction":{"id":"6581c785703b09b8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:34:34.635+01:00","message":"Kibana process configured with roles: [background_tasks, ui]","log":{"level":"INFO","logger":"node"},"process":{"pid":21904},"trace":{"id":"59e4502d810078491b8b839d845577ea"},"transaction":{"id":"6581c785703b09b8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-26T14:41:22.867+01:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"total_all_statuses\":0,\"updating\":0},\"fleet_server\":{\"total_all_statuses\":0,\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"updating\":0,\"num_host_urls\":0}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":20718},"trace":{"id":"d23b454315a17b7c2f968f35a7ac3445"},"transaction":{"id":"6b7005d4051b610e"}}

Here is Journalctl -u kibana.service log

Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.918+01:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.979+01:00][INFO ][plugins.synthetics] Installed synthetics index templates
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.001+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: scheduled with interval 1h
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.455+01:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/screenshotting/chr
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.629+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.691+01:00][INFO ][status] Kibana is now available (was degraded)
Mar 26 14:26:22 cn11575apld001 kibana[20718]: [2023-03-26T14:26:22.798+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":
Mar 26 14:41:22 cn11575apld001 kibana[20718]: [2023-03-26T14:41:22.867+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":
Mar 26 14:56:22 cn11575apld001 kibana[20718]: [2023-03-26T14:56:22.919+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":

Here is the curl status for kibana

[devopsadmin@cn11575apld001 ~]$ sudo curl -XGET http://10.66.36.44:5601/ -I
HTTP/1.1 302 Found
location: /login?next=%2F
x-content-type-options: nosniff
referrer-policy: no-referrer-when-downgrade
content-security-policy: script-src 'self' 'unsafe-eval'; worker-src blob: 'self'; style-src 'unsafe-inline' 'self'
kbn-name: kibana-monitoring
kbn-license-sig: ebbec1d966d0c7e52e8295dbf6b07eb145e9a03b8fe1e3aee542b200ebfc7d3c
cache-control: private, no-cache, no-store, must-revalidate
content-length: 0
Date: Sun, 26 Mar 2023 13:57:55 GMT
Connection: keep-alive
Keep-Alive: timeout=120

Port status :

[devopsadmin@cn11575apld001 ~]$ sudo lsof -i:5601
COMMAND   PID   USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
node    20718 kibana   57u  IPv4 5955954      0t0  TCP cn11575apld001:esmagent (LISTEN)

Service Status here

[devopsadmin@cn11575apld001 ~]$ sudo systemctl status kibana.service
● kibana.service - Kibana
   Loaded: loaded (/usr/lib/systemd/system/kibana.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2023-03-26 14:21:49 BST; 37min ago
     Docs: https://www.elastic.co
 Main PID: 20718 (node)
   CGroup: /system.slice/kibana.service
           └─20718 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist

Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.906+01:00][INFO ][plugins.fleet] Fleet setup completed
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.918+01:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask
Mar 26 14:22:10 cn11575apld001 kibana[20718]: [2023-03-26T14:22:10.979+01:00][INFO ][plugins.synthetics] Installed synthetics index templates
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.001+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: scheduled with interval 1h
Mar 26 14:22:11 cn11575apld001 kibana[20718]: [2023-03-26T14:22:11.455+01:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/scr...adless_shell
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.629+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
Mar 26 14:22:13 cn11575apld001 kibana[20718]: [2023-03-26T14:22:13.691+01:00][INFO ][status] Kibana is now available (was degraded)
Mar 26 14:26:22 cn11575apld001 kibana[20718]: [2023-03-26T14:26:22.798+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealth...
Mar 26 14:41:22 cn11575apld001 kibana[20718]: [2023-03-26T14:41:22.867+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealth...
Mar 26 14:56:22 cn11575apld001 kibana[20718]: [2023-03-26T14:56:22.919+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealth...
Hint: Some lines were ellipsized, use -l to show in full.

Still i'm unable to bring up kibana in browser. Kinldy help me on this .

Thanks in Advance.

Thanks,
Siva

Hi @sivamca127 Welcome to the community.

Looks like you have made good progress.

What is the error in the browser?

Did you try different browser?

What you posted looks pretty good.

Did you do anything like a proxy (not needed just asking)

Hi @stephenb,

Currently i'm using Microsoft Edge and Chrome Browsers.

Error Message: The site was not reachable (http://my-host-Ip:5601)

i'm not doing with proxy.

Thanks,
Siva

So you tried the same IP as the curl above when you got the 302?

From the same machine?

Or from a different machine... Can you curl from the same machine you are trying to run Kibana.

http://10.66.36.44:5601/

And try another browser

Hi @stephenb ,

Yes I'm trying from same machine where elasticsearch and kibana configured.

Thanks,
Siva

It would be very unusual if curl worked and a browser did not.

The curl command responded as I would expect.

The site is unreachable is generally a connectivity issue.

Try Chrome or FireFox

If it still does not work, you will need to show logs from the Browser Dev Tools.

Also do you see anything in the Kibana logs when you try to connect?

Hi @stephenb,

I've tried from chrome and here is the developer console result

FYI , I've used Java17

"use strict";
// Copyright 2012 The Chromium Authors
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
/**
* @fileoverview
* NOTE: This file is deprecated, and provides only the minimal LoadTimeData
* functions for places in the code still not using JS modules. Use
* load_time_data.ts in all new code.
*
* This file defines a singleton which provides access to all data
* that is available as soon as the page's resources are loaded (before DOM
* content has finished loading). This data includes both localized strings and
* any data that is important to have ready from a very early stage (e.g. things
* that must be displayed right away).
*
* Note that loadTimeData is not guaranteed to be consistent between page
* refreshes (https://crbug.com/740629) and should not contain values that might
* change if the page is re-opened later.
*/
/** @type {!LoadTimeData} */
// eslint-disable-next-line no-var
var loadTimeData;
class LoadTimeData {
    constructor() {
        /** @type {?Object} */
        this.data_ = null;
    }
    /**
     * Sets the backing object.
     *
     * Note that there is no getter for |data_| to discourage abuse of the form:
     *
     *     var value = loadTimeData.data()['key'];
     *
     * @param {Object} value The de-serialized page data.
     */
    set data(value) {
        expect(!this.data_, 'Re-setting data.');
        this.data_ = value;
    }
    /**
     * @param {string} id An ID of a value that might exist.
     * @return {boolean} True if |id| is a key in the dictionary.
     */
    valueExists(id) {
        return id in this.data_;
    }
    /**
     * Fetches a value, expecting that it exists.
     * @param {string} id The key that identifies the desired value.
     * @return {*} The corresponding value.
     */
    getValue(id) {
        expect(this.data_, 'No data. Did you remember to include strings.js?');
        const value = this.data_[id];
        expect(typeof value !== 'undefined', 'Could not find value for ' + id);
        return value;
    }
    /**
     * As above, but also makes sure that the value is a string.
     * @param {string} id The key that identifies the desired string.
     * @return {string} The corresponding string value.
     */
    getString(id) {
        const value = this.getValue(id);
        expectIsType(id, value, 'string');
        return /** @type {string} */ (value);
    }
    /**
     * Returns a formatted localized string where $1 to $9 are replaced by the
     * second to the tenth argument.
     * @param {string} id The ID of the string we want.
     * @param {...(string|number)} var_args The extra values to include in the
     *     formatted output.
     * @return {string} The formatted string.
     */
    getStringF(id, var_args) {
        const value = this.getString(id);
        if (!value) {
            return '';
        }
        const args = Array.prototype.slice.call(arguments);
        args[0] = value;
        return this.substituteString.apply(this, args);
    }
    /**
     * Returns a formatted localized string where $1 to $9 are replaced by the
     * second to the tenth argument. Any standalone $ signs must be escaped as
     * $$.
     * @param {string} label The label to substitute through.
     *     This is not an resource ID.
     * @param {...(string|number)} var_args The extra values to include in the
     *     formatted output.
     * @return {string} The formatted string.
     */
    substituteString(label, var_args) {
        const varArgs = arguments;
        return label.replace(/\$(.|$|\n)/g, function (m) {
            expect(m.match(/\$[$1-9]/), 'Unescaped $ found in localized string.');
            return m === '$$' ? '$' : varArgs[m[1]];
        });
    }
    /**
     * As above, but also makes sure that the value is a boolean.
     * @param {string} id The key that identifies the desired boolean.
     * @return {boolean} The corresponding boolean value.
     */
    getBoolean(id) {
        const value = this.getValue(id);
        expectIsType(id, value, 'boolean');
        return /** @type {boolean} */ (value);
    }
    /**
     * As above, but also makes sure that the value is an integer.
     * @param {string} id The key that identifies the desired number.
     * @return {number} The corresponding number value.
     */
    getInteger(id) {
        const value = this.getValue(id);
        expectIsType(id, value, 'number');
        expect(value === Math.floor(value), 'Number isn\'t integer: ' + value);
        return /** @type {number} */ (value);
    }
    /**
     * Override values in loadTimeData with the values found in |replacements|.
     * @param {Object} replacements The dictionary object of keys to replace.
     */
    overrideValues(replacements) {
        expect(typeof replacements === 'object', 'Replacements must be a dictionary object.');
        for (const key in replacements) {
            this.data_[key] = replacements[key];
        }
    }
}
/**
* Checks condition, throws error message if expectation fails.
* @param {*} condition The condition to check for truthiness.
* @param {string} message The message to display if the check fails.
*/
function expect(condition, message) {
    if (!condition) {
        throw new Error('Unexpected condition on ' + document.location.href + ': ' + message);
    }
}
/**
* Checks that the given value has the given type.
* @param {string} id The id of the value (only used for error message).
* @param {*} value The value to check the type on.
* @param {string} type The type we expect |value| to be.
*/
function expectIsType(id, value, type) {
    expect(typeof value === type, '[' + value + '] (' + id + ') is not a ' + type);
}
expect(!loadTimeData, 'should only include this file once');
loadTimeData = new LoadTimeData();
// Expose |loadTimeData| directly on |window|, since within a JS module the
// scope is local and not all files have been updated to import the exported
// |loadTimeData| explicitly.
window.loadTimeData = loadTimeData;
**console.warn('crbug/1173575, non-JS module files deprecated.');**

Thanks,
Siva

Hi @stephenb ,

Here is the curl status:

[devopsadmin@cn11575apld001 ~]$ curl -XGET http://10.66.36.44:5601 -v
* About to connect() to 10.66.36.44 port 5601 (#0)
*   Trying 10.66.36.44...
* Connected to 10.66.36.44 (10.66.36.44) port 5601 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.66.36.44:5601
> Accept: */*
>
< HTTP/1.1 302 Found
< location: /login?next=%2F
< x-content-type-options: nosniff
< referrer-policy: no-referrer-when-downgrade
< content-security-policy: script-src 'self' 'unsafe-eval'; worker-src blob: 'self'; style-src 'unsafe-inline' 'self'
< kbn-name: kibana-monitoring
< kbn-license-sig: ebbec1d966d0c7e52e8295dbf6b07eb145e9a03b8fe1e3aee542b200ebfc7d3c
< cache-control: private, no-cache, no-store, must-revalidate
< content-length: 0
< Date: Mon, 27 Mar 2023 10:36:27 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
<
* Connection #0 to host 10.66.36.44 left intact

Thanks,
Siva

@sivamca127

Apologies I Meant the console logs from the Browser.

Can you provide a screenshot of the main browser with main error.

Like you did in the first post for elasticsearch but this time with Kibana

I do not know what the issue is... If curl works the browser should...

Hi @stephenb ,

Here is the screenshot .

FYI

Also here is status for kibana from server side

FYI

[devopsadmin@cn11575apld001 ~]$ sudo /usr/share/kibana/bin/kibana --allow-root
[2023-03-27T12:38:30.410+01:00][INFO ][node] Kibana process configured with roles: [background_tasks, ui]
[2023-03-27T12:38:45.961+01:00][INFO ][plugins-service] Plugin "cloudChat" is disabled.
[2023-03-27T12:38:45.961+01:00][INFO ][plugins-service] Plugin "cloudExperiments" is disabled.
[2023-03-27T12:38:45.961+01:00][INFO ][plugins-service] Plugin "cloudFullStory" is disabled.
[2023-03-27T12:38:45.961+01:00][INFO ][plugins-service] Plugin "cloudGainsight" is disabled.
[2023-03-27T12:38:45.977+01:00][INFO ][plugins-service] Plugin "profiling" is disabled.
[2023-03-27T12:38:46.087+01:00][INFO ][http.server.Preboot] http server running at http://10.66.36.44:5601
[2023-03-27T12:38:46.177+01:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2023-03-27T12:38:46.259+01:00][WARN ][config.deprecation] The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set "xpack.reporting.roles.enabled" to "false" to adopt the future behavior before upgrading.
[2023-03-27T12:38:46.654+01:00][INFO ][plugins-system.standard] Setting up [127] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,fieldFormats,expressions,screenshotting,dataViews,embeddable,uiActionsEnhanced,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,spaces,security,snapshotRestore,lists,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,console,bfetch,data,watcher,reporting,fileUpload,ingestPipelines,alerting,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,eventAnnotation,dataViewFieldEditor,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,discover,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,dataVisualizer,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,osquery,observability,aiops,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement]
[2023-03-27T12:38:46.675+01:00][INFO ][plugins.taskManager] TaskManager is identified by the Kibana UUID: 56632266-281d-4fd4-9fcf-548e8517768b
[2023-03-27T12:38:46.794+01:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-03-27T12:38:46.795+01:00][WARN ][plugins.security.config] Session cookies will be transmitted over insecure connections. This is not recommended.
[2023-03-27T12:38:46.827+01:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-03-27T12:38:46.828+01:00][WARN ][plugins.security.config] Session cookies will be transmitted over insecure connections. This is not recommended.
[2023-03-27T12:38:46.842+01:00][WARN ][plugins.encryptedSavedObjects] Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-03-27T12:38:46.855+01:00][WARN ][plugins.actions] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-03-27T12:38:46.862+01:00][INFO ][plugins.notifications] Email Service Error: Email connector not specified.
[2023-03-27T12:38:46.995+01:00][WARN ][plugins.reporting.config] Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-03-27T12:38:47.003+01:00][WARN ][plugins.alerting] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-03-27T12:38:47.063+01:00][INFO ][plugins.ruleRegistry] Installing common resources shared between all indices
[2023-03-27T12:38:47.126+01:00][INFO ][plugins.cloudSecurityPosture] Registered task successfully [Task: cloud_security_posture-stats_task]
[2023-03-27T12:38:47.994+01:00][WARN ][plugins.screenshotting.config] Chromium sandbox provides an additional layer of protection, but is not supported for Linux Red Hat Linux 7.5 OS. Automatically setting 'xpack.screenshotting.browser.chromium.disableSandbox: true'.
[2023-03-27T12:38:48.218+01:00][INFO ][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
[2023-03-27T12:38:48.219+01:00][INFO ][savedobjects-service] Starting saved objects migrations
[2023-03-27T12:38:48.293+01:00][INFO ][savedobjects-service] [.kibana] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 37ms.
[2023-03-27T12:38:48.296+01:00][INFO ][savedobjects-service] [.kibana_task_manager] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 33ms.
[2023-03-27T12:38:48.299+01:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 6ms.
[2023-03-27T12:38:48.304+01:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 8ms.
[2023-03-27T12:38:48.308+01:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 9ms.
[2023-03-27T12:38:48.311+01:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 7ms.
[2023-03-27T12:38:48.313+01:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> CHECK_TARGET_MAPPINGS. took: 5ms.
[2023-03-27T12:38:48.315+01:00][INFO ][savedobjects-service] [.kibana] CHECK_TARGET_MAPPINGS -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 2ms.
[2023-03-27T12:38:48.315+01:00][INFO ][savedobjects-service] [.kibana] CHECK_VERSION_INDEX_READY_ACTIONS -> DONE. took: 0ms.
[2023-03-27T12:38:48.316+01:00][INFO ][savedobjects-service] [.kibana] Migration completed after 60ms
[2023-03-27T12:38:48.317+01:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> CHECK_TARGET_MAPPINGS. took: 6ms.
[2023-03-27T12:38:48.318+01:00][INFO ][savedobjects-service] [.kibana_task_manager] CHECK_TARGET_MAPPINGS -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 1ms.
[2023-03-27T12:38:48.319+01:00][INFO ][savedobjects-service] [.kibana_task_manager] CHECK_VERSION_INDEX_READY_ACTIONS -> DONE. took: 1ms.
[2023-03-27T12:38:48.319+01:00][INFO ][savedobjects-service] [.kibana_task_manager] Migration completed after 56ms
[2023-03-27T12:38:48.325+01:00][INFO ][plugins-system.preboot] Stopping all plugins.
[2023-03-27T12:38:48.326+01:00][INFO ][plugins-system.standard] Starting [127] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,fieldFormats,expressions,screenshotting,dataViews,embeddable,uiActionsEnhanced,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,spaces,security,snapshotRestore,lists,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,console,bfetch,data,watcher,reporting,fileUpload,ingestPipelines,alerting,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,eventAnnotation,dataViewFieldEditor,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,discover,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,dataVisualizer,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,osquery,observability,aiops,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement]
[2023-03-27T12:38:50.064+01:00][INFO ][plugins.fleet] Task Fleet-Usage-Sender-1.1.0 scheduled with interval 1h
[2023-03-27T12:38:50.127+01:00][INFO ][plugins.monitoring.monitoring] config sourced from: production cluster
[2023-03-27T12:38:52.606+01:00][INFO ][http.server.Kibana] http server running at http://10.66.36.44:5601
[2023-03-27T12:38:52.635+01:00][INFO ][plugins.fleet] Task Fleet-Usage-Logger-Task scheduled with interval 15m
[2023-03-27T12:38:52.718+01:00][INFO ][status] Kibana is now degraded
[2023-03-27T12:38:52.853+01:00][INFO ][plugins.monitoring.monitoring.kibana-monitoring] Starting monitoring stats collection
[2023-03-27T12:38:52.854+01:00][INFO ][plugins.fleet] Beginning fleet setup
[2023-03-27T12:38:52.913+01:00][INFO ][plugins.ruleRegistry] Installed common resources shared between all indices
[2023-03-27T12:38:52.914+01:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.uptime.alerts
[2023-03-27T12:38:52.915+01:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-security.alerts
[2023-03-27T12:38:52.916+01:00][INFO ][plugins.ruleRegistry] Installing resources for index .preview.alerts-security.alerts
[2023-03-27T12:38:52.917+01:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.logs.alerts
[2023-03-27T12:38:52.917+01:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.metrics.alerts
[2023-03-27T12:38:52.918+01:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.apm.alerts
[2023-03-27T12:38:52.986+01:00][INFO ][plugins.ruleRegistry] Installed resources for index .preview.alerts-security.alerts
[2023-03-27T12:38:52.992+01:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.metrics.alerts
[2023-03-27T12:38:52.993+01:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-security.alerts
[2023-03-27T12:38:52.994+01:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.logs.alerts
[2023-03-27T12:38:52.995+01:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.apm.alerts
[2023-03-27T12:38:52.997+01:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.uptime.alerts
[2023-03-27T12:38:53.024+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: scheduled with interval 1h
[2023-03-27T12:38:53.183+01:00][INFO ][plugins.fleet] Fleet setup completed
[2023-03-27T12:38:53.204+01:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask
[2023-03-27T12:38:53.268+01:00][INFO ][plugins.synthetics] Installed synthetics index templates
[2023-03-27T12:38:53.851+01:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/screenshotting/chromium/headless_shell-linux_x64/headless_shell
[2023-03-27T12:38:55.787+01:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
[2023-03-27T12:38:58.717+01:00][INFO ][status] Kibana is now available (was degraded)
[2023-03-27T12:43:49.939+01:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}

Thanks,
Siva

That's a simple timeout which means it can't be reached. If you're telling me curl to works on the same box with the same IP/ URL and that returns 302, and the exact same in a browser times out....I have no idea.

Try

http://10.66.36.44:5601/login?next=%2F

Another thing you can try. Perhaps you have some sort of company default or something.

Change the serve port to 80 or 8080

server.port: 8080

Hi @stephenb ,

I've done as mentioned above but no luck. Still i'm getting connection error like below with port 5601, 80 and 8080

Thanks,
Siva

Did you try this.

Apologies I have no explanation, this is most likely something with your local environment.

This a connection issue... again if curl works from the exact same box with the exact same IP / URL and returns the 302 I am out of thoughts at this point.

try setting in kibana.yml

server.host: 0.0.0.0

server.host
This setting specifies the host of the back end server. To allow remote users to connect, set the value to the IP address or DNS name of the Kibana server. Use 0.0.0.0 to make Kibana listen on all IPs (public and private). Default: "localhost"

have you tried

http://localhost:5601

Uninstall / Re-Install?

@dadoonet Any thoughts?

Hi @stephenb ,

I've tried http://localhost:5601 using curl .
FYI

[devopsadmin@cn11575apld001 ~]$ sudo curl http://localhost:5601 -v
* About to connect() to localhost port 5601 (#0)
*   Trying ::1...
* Connection refused
*   Trying 127.0.0.1...
* Connection refused
* Failed connect to localhost:5601; Connection refused
* Closing connection 0
curl: (7) Failed connect to localhost:5601; Connection refused

I'm working with my internal networking team on the same. I'll update the thread once i'm done with discussion.

Thanks,
Siva

Hi @stephenb ,

I've reached my network team and everything looks good and we had a confirmation from Microsoft about the firewall. they are asking me to get some additional help from your side. Can you please provide your insights on the same?

Thanks,
Siva

Hi @sivamca127

This is a community forum staffed by volunteers, not paid support we have given you many ideas we do not directly engage with users.

Try on another server...

This is most likely related to something in your environment...

Try commenting on these out and restart kibana

# server.port: 5601
# server.host: "10.66.36.44"

Then try from the same host kibana is on try this in the web browser on the same host kibana is on, not a different host.

http://localhost:5601

My suspicion is that your browser is not even getting to through to Kibana, so there is nothing to gain from looking at the Kibana configuration. But that is just a suspicion. We need to be sure, and since your Kibana instance is running on RHEL it is time to break out some low level tools!

Install tcpdump on your RHEL box, and learn how to capture all TCP traffic to & from port 5601. Run tcpdump in one terminal window, and use curl from another to connect to http://10.66.36.44:5601 like this:

curl --verbose --noproxy '*' http://10.66.36.44:5601

Now watch the output from tcpdump and see what curl is saying to Kibana and what Kibana is saying back to curl.

Abd then you repeat the exercise, but this time from your browser. Do you see anything from tcpdump, or is it now silent? If it is silent then you have the proof that your browser is not talking to Kibana but is busy trying to talk to a proxy server, busy trying to drill a hole through a firewall, traffic being sent the wrong way by a router, etc etc. If you do see traffic from your browser reaching Kibana, then we need to look closely at what tcpdump is displaying, but that is step no. 2. Step 1 is proving that your browser is reaching Kibana.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.