Kibana not starting on server1

Elastic Community and Ecosystem Elastic Training
1

Moved here from last thread:

I can now access the environment, but Kibana is not working. Elasticsearch is started and responds fine, but when trying to start Kibana from a fresh tar extraction, I am getting:

log [15:39:04.736] [fatal][root] { Error: Request Timeout after 30000ms
at /home/elastic/kibana-7.3.1-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:362:15
at Timeout. (/home/elastic/kibana-7.3.1-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:391:7)
at ontimeout (timers.js:436:11)
at tryOnTimeout (timers.js:300:5)
at listOnTimeout (timers.js:263:5)
at Timer.processTimers (timers.js:223:10)
status: undefined,
displayName: 'RequestTimeout',
message: 'Request Timeout after 30000ms',
body: undefined,
isBoom: true,
isServer: true,
data: null,
output:
{ statusCode: 503,
payload:
{ statusCode: 503,
error: 'Service Unavailable',
message: 'Request Timeout after 30000ms' },
headers: {} },
reformat: [Function],
[Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }
log [15:39:04.739] [debug][server] stopping server

I can cURL the elastic service fine on localhost, server1 and the given amazon instance host name. They all respond on port 9200. Node-name of the instance is node1.

Kibana was started with the kibana --host=0.0.0.0, as per the lab instructions.

2

To follow up:

I querried elastic via:

curl -X GET "http://localhost:9200"
curl -X GET "http://server1:9200"
curl -X GET "http://theExtraLongAmazonWebInstanceHostname:9200"

and opened the amazon instance in chrome on :9200.

Result: All of them return the elastic standard message ("You know, for search").

I also tried setting elasticsearch.hosts to the relevant hosts in kibana.yml.
Same result every time.

3

Hi Marcus, there is definitely something wrong going on :slight_smile: Resume your lab and send me a private message containing the new instance URL (extraLongAmazon...) and will take a look. Again, sorry for the bad experience. It looks like you got the short straw :confused:

4

Hm yes. The Strigo instance is not really working for me. Today, I cannot even get elasticsearch running anymore. It forever tries to negotiate itself as cluster master.

Is there perhaps a chance to get the datasets, so I can learn offline? I need to get through Elasticsearch Engineer 1+2 in the next 4 weeks. All that bug hunting really eats up my time.

5

Hi there,

Unfortunately, I am having a similar issue. I am able to access the environment but Kibana is not working. I have tried to use a fresh tar extraction and cleared all my internet cookies.

I get the following error messages:

[elastic@server1 ~] ls -l total 8 drwxrwxr-x 1 elastic elastic 4096 Aug 11 09:00 elasticsearch drwxrwxr-x 1 elastic elastic 4096 Aug 11 08:55 kibana [elastic@server1 ~] ./kibana/bin/kibana
log [13:55:17.605] [warning][plugins-discovery] Expect plugin "id" in camelCase, but found: apm_oss
log [13:55:17.631] [warning][plugins-discovery] Expect plugin "id" in camelCase, but found: triggers_actions_ui
log [13:55:27.396] [info][plugins-service] Plugin "visTypeXy" is disabled.
log [13:55:27.396] [info][plugins-service] Plugin "endpoint" is disabled.
log [13:55:27.397] [info][plugins-service] Plugin "ingestManager" is disabled.
log [13:55:27.397] [info][plugins-service] Plugin "lists" is disabled.
log [13:55:29.134] [info][plugins-system] Setting up [94] plugins: [taskManager,licensing,observability,eventLog,encryptedSavedObjects,code,kibanaLegacy,devTools,usageCollection,ossTelemetry,telemetryCollectionManager,telemetry,telemetryCollectionXpack,translations,uiActions,statusPage,share,newsfeed,mapsLegacy,mapsLegacyLicensing,kibanaUtils,inspector,embeddable,advancedUiActions,embeddableEnhanced,drilldowns,kibanaReact,indexPatternManagement,esUiShared,discover,charts,bfetch,expressions,data,home,cloud,console,consoleExtensions,apm_oss,searchprofiler,painlessLab,grokdebugger,management,upgradeAssistant,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,reporting,licenseManagement,watcher,advancedSettings,telemetryManagementSection,fileUpload,dataEnhanced,savedObjects,visualizations,visTypeVega,visTypeVislib,visTypeTimeseries,rollup,visTypeTagcloud,visTypeTimelion,features,security,snapshotRestore,transform,ingestPipelines,canvas,visTypeMetric,visTypeTable,visTypeMarkdown,inputControlVis,navigation,lens,graph,maps,visualize,dashboard,dashboardEnhanced,savedObjectsManagement,spaces,actions,case,alerting,alertingBuiltins,triggers_actions_ui,infra,monitoring,logstash,uptime,ml,siem,apm]
log [13:55:29.145] [warning][config][encryptedSavedObjects][plugins] Generating a random key for xpack.encryptedSavedObjects.encryptionKey. To be able to decrypt encrypted saved objects attributes after restart, please set xpack.encryptedSavedObjects.encryptionKey in kibana.yml
log [13:55:29.361] [warning][config][plugins][security] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml
log [13:55:29.361] [warning][config][plugins][security] Session cookies will be transmitted over insecure connections. This is not recommended.
log [13:55:29.431] [warning][actions][actions][plugins] APIs are disabled due to the Encrypted Saved Objects plugin using an ephemeral encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in kibana.yml.
log [13:55:29.449] [warning][alerting][alerting][plugins][plugins] APIs are disabled due to the Encrypted Saved Objects plugin using an ephemeral encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in kibana.yml.
log [13:55:29.485] [info][monitoring][monitoring][plugins] config sourced from: production cluster
log [13:55:29.486] [warning][monitoring][monitoring][plugins] X-Pack Monitoring Cluster Alerts will not be available: undefined
log [13:55:29.500] [error][elasticsearch][monitoring] Request error, retrying
GET http://server1:9200/_xpack => connect ECONNREFUSED 172.18.0.2:9200
log [13:55:29.503] [error][elasticsearch][monitoring] Request error, retrying
GET http://server1:9200/_xpack => connect ECONNREFUSED 172.18.0.2:9200
log [13:55:29.504] [error][data][elasticsearch] Request error, retrying
GET http://server1:9200/_xpack => connect ECONNREFUSED 172.18.0.2:9200
log [13:55:29.506] [warning][elasticsearch][monitoring] Unable to revive connection: http://server1:9200/
log [13:55:29.506] [warning][elasticsearch][monitoring] No living connections
log [13:55:29.507] [warning][licensing][plugins] License information could not be obtained from Elasticsearch due to Error: No Living connections error
log [13:55:29.777] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
log [13:55:29.779] [warning][elasticsearch][monitoring] Unable to revive connection: http://server1:9200/
log [13:55:29.779] [warning][elasticsearch][monitoring] No living connections
log [13:55:29.779] [warning][licensing][plugins] License information could not be obtained from Elasticsearch due to Error: No Living connections error
log [13:55:29.780] [warning][data][elasticsearch] Unable to revive connection: http://server1:9200/
log [13:55:29.781] [warning][data][elasticsearch] No living connections
log [13:55:29.781] [warning][licensing][plugins] License information could not be obtained from Elasticsearch due to Error: No Living connections error
log [13:55:29.782] [info][plugins][searchprofiler] You cannot use searchprofiler because license information is not available at this time.
log [13:55:29.782] [info][painlessLab][plugins] You cannot use painlessLab because license information is not available at this time.
log [13:55:29.783] [info][indexManagement][plugins] You cannot use index_management because license information is not available at this time.
log [13:55:29.783] [info][plugins][remoteClusters] You cannot use Remote Clusters because license information is not available at this time.
log [13:55:29.784] [info][crossClusterReplication][plugins] You cannot use crossClusterReplication because license information is not available at this time.
log [13:55:29.784] [info][indexLifecycleManagement][plugins] You cannot use index_lifecycle_management because license information is not available at this time.
log [13:55:29.785] [info][plugins][watcher] You cannot use watcher because license information is not available at this time.
log [13:55:29.785] [info][plugins][rollup] You cannot use rollup because license information is not available at this time.
log [13:55:29.786] [info][plugins][snapshotRestore] You cannot use snapshot_restore because license information is not available at this time.
log [13:55:29.786] [info][plugins][transform] You cannot use transform because license information is not available at this time.
log [13:55:29.786] [info][ingestPipelines][plugins] You cannot use ingest_pipelines because license information is not available at this time.
log [13:55:29.787] [info][kibana-monitoring][monitoring][monitoring][plugins] Monitoring status upload endpoint is not enabled in Elasticsearch:Monitoring stats collection is stopped
log [13:55:29.792] [error][admin][elasticsearch] Request error, retrying
GET http://server1:9200/_nodes?filter_path=nodes..version%2Cnodes..http.publish_address%2Cnodes.*.ip => connect ECONNREFUSED 172.18.0.2:9200
log [13:55:29.793] [warning][admin][elasticsearch] Unable to revive connection: http://server1:9200/
log [13:55:29.794] [warning][admin][elasticsearch] No living connections
log [13:55:29.795] [error][savedobjects-service] Unable to retrieve version information from Elasticsearch nodes.
log [13:55:32.293] [warning][admin][elasticsearch] Unable to revive connection: http://server1:9200/
log [13:55:32.294] [warning][admin][elasticsearch] No living connections
log [13:55:34.796] [warning][admin][elasticsearch] Unable to revive connection: http://server1:9200/
log [13:55:34.797] [warning][admin][elasticsearch] No living connections
log [13:55:37.298] [warning][admin][elasticsearch] Unable to revive connection: http://server1:9200/
log [13:55:37.299] [warning][admin][elasticsearch] No living connections
log [13:55:39.801] [warning][admin][elasticsearch] Unable to revive connection: http://server1:9200/
log [13:55:39.802] [warning][admin][elasticsearch] No living connection

I also tried to make changes in Kibana.yml but was denied access.
Grateful if someone could help please.

Thanks
Harkish

6

Having the same issue Here, I can find lots of questions on the same issue but there is not a single answer on that!

7

Hi vazma, thank you for your message and I am sorry that you are experience issues. Can you please open a new question detailing your problem? It is always good to state, the course name, the module name, the lab step, and the issue you are seeing.

Thank you

1 Like