Kibana 7.4.0 on windows - command prompt - not able to start

Elastic Stack Kibana
1

Hi,

I am new to the Elastic Stack, I have ElasticSearch 7.4.0 installed and started successful at the command prompt on Windows server 2016.

I am able to go to http://127.0.0.1:9200/_cat/health?v and I see status green.
I also installed cerebro 0.8.4 and successfully connect to elastic search and also shows green

Then I attempted to start Kirbana 7.4.0 at the command prompt (all on the same machine), then Kirbana does not start successful, it throw errors and exceptions; does anyone know about this? any suggestion-advice? perhaps there is a configuration I am missing. Below is the error for your reference, Thanks in advance for your help.

  log   [15:33:59.171] [info][migrations] Creating index .kibana_task_manager_1.
Could not create APM Agent configuration: Request Timeout after 30000ms
 error  [15:34:28.348] [warning][process] UnhandledPromiseRejectionWarning: Error: Request Timeout after 30000ms
    at C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:397:9
    at Timeout.<anonymous> (C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:429:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
    at emitWarning (internal/process/promises.js:81:15)
    at emitPromiseRejectionWarnings (internal/process/promises.js:120:9)
    at process._tickCallback (internal/process/next_tick.js:69:34)
 error  [15:34:28.352] [warning][process] Error: Request Timeout after 30000ms
    at C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:397:9
    at Timeout.<anonymous> (C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:429:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
  log   [15:34:29.176] [error][status][plugin:spaces@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [15:34:29.180] [fatal][root] { Error: Request Timeout after 30000ms
    at C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:397:9
    at Timeout.<anonymous> (C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:429:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
  status: undefined,
  displayName: 'RequestTimeout',
  message: 'Request Timeout after 30000ms',
  body: undefined,
  isBoom: true,
  isServer: true,
  data: null,
  output:
   { statusCode: 503,
     payload:
      { statusCode: 503,
        error: 'Service Unavailable',
        message: 'Request Timeout after 30000ms' },
     headers: {} },
  reformat: [Function],
  [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }
  log   [15:34:29.184] [info][plugins-system] Stopping all plugins.
  log   [15:34:29.185] [info][data][plugins] Stopping plugin
  log   [15:34:29.186] [info][plugins][translations] Stopping plugin
  log   [15:34:29.187] [info][plugins][security] Stopping plugin

 FATAL  Error: Request Timeout after 30000ms
1 Like
2

@LeeDr can u please take a look ?

Thanks
Rashmi

3

Hi. Let's start by double-checking exactly which downloads you have for both Elasticsearch and Kibana. Are they the default distributions;

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.0-windows-x86_64.zip
and
https://artifacts.elastic.co/downloads/kibana/kibana-7.4.0-windows-x86_64.zip

Or did you get the pure OSS distribution of either of them like (look for the oss in the zip file name);
https://artifacts.elastic.co/downloads/kibana/kibana-oss-7.4.0-windows-x86_64.zip
or
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-7.4.0-windows-x86_64.zip

If you have the default distribution for both, then they should work together fine. I just downloaded them both, unzipped them, started Elasticsearch, and started Kibana with no problems.

From your Kibana log, it looks like Kibana can reach your Elasticsearch but can't create an index. My first thought on that is that maybe you're over the disk.watermark.high setting.
https://www.elastic.co/guide/en/elasticsearch/reference/current/disk-allocator.html
But I forced my Elasticsearch into that state and the Kibana logs are different.

What does your Elasticsearch console output show when Kibana is logging the errors?

Can you paste the output of these requests (I'll show my responses. I'm using curl but you can use a browser or other tools);

curl http://localhost:9200
{
  "name" : "LEEDR-XPS",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "p9bFrOu9RHGtiF-moLJgMw",
  "version" : {
    "number" : "7.4.0",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "22e1767283e61a198cb4db791ea66e3f11ab9910",
    "build_date" : "2019-09-27T08:36:48.569419Z",
    "build_snapshot" : false,
    "lucene_version" : "8.2.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

curl http://127.0.0.1:9200/_cat/health?v
epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1571245129 16:58:49  elasticsearch green           1         1      3   3    0    0        0             0                  -                100.0%

curl http://127.0.0.1:9200/_cat/indices?v
health status index                    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana_task_manager_1   C4F9O-D9R4-OMJ7CXJOeuQ   1   0          2            0     12.1kb         12.1kb
green  open   .apm-agent-configuration DZ9K7Iu0Ssy3gf0VFJobVg   1   0          0            0       283b           283b
green  open   .kibana_1                4FAHdDSQT8-MggfgWjrVgA   1   0          1            0      5.9kb          5.9kb

Do you have anything at all in your elasticsearch.yml or kibana.yml? You shouldn't have to set anything in them to get to this initial point of functionality.

Regards,
Lee

4

Hi Lee,

These are the download files:

  • elaticsearch-7.4.0-windows-x86_64.zip
  • kibana-7.4.0-windows-86_64.zip

I guess, the above is the default distribution from what you mentioned.

When going to http://127.0.01:9200, I get the following output:

{
  "name" : "LOGCOLLECTOR",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "QoiuWNEGTPuAMqe-H0zXMQ",
  "version" : {
    "number" : "7.4.0",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "22e1767283e61a198cb4db791ea66e3f11ab9910",
    "build_date" : "2019-09-27T08:36:48.569419Z",
    "build_snapshot" : false,
    "lucene_version" : "8.2.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

For http://127.0.0.1:9200/_cat/health?v, I get:

epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1571250904 18:35:04  elasticsearch green           1         1      1   1    0    0        0             1                  -                100.0%

For http://127.0.0.1:9200/_cat/indices?v, I get the following:

health status index                    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .apm-agent-configuration vJgFf1mzRVyMxeIU-OAICw   1   0          0            0       283b           283b

I don't see any Kibana output like in your case, I guess because Kibana is not able to start, or to create those indices entries.

5

Also, at first I thought that Kibana is not able to log into Elastic Search because of user/pass; so that I uncommented user/pass in kibana.yml ; but it still did not work; so that I commented back out as original and still does not work.

Here is the log error (elasticsearch.yml and kibana.yml unchanged, as original with everything commented out):

  log   [17:18:01.487] [info][migrations] Creating index .kibana_task_manager_1.
  log   [17:18:01.493] [info][migrations] Creating index .kibana_1.
  log   [17:18:30.687] [warning][license][xpack] License information from the X-Pack plugin could not be obtained from Elasticsearch for the [data] cluster. Error: Request Timeout after 30000ms
  log   [17:18:30.690] [error][status][plugin:xpack_main@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.692] [error][status][plugin:graph@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.693] [error][status][plugin:spaces@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.694] [error][status][plugin:searchprofiler@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.695] [error][status][plugin:ml@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.695] [error][status][plugin:tilemap@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.696] [error][status][plugin:watcher@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.697] [error][status][plugin:grokdebugger@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.698] [error][status][plugin:logstash@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.699] [error][status][plugin:beats_management@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.700] [error][status][plugin:maps@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.700] [error][status][plugin:index_management@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.703] [error][status][plugin:index_lifecycle_management@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.705] [error][status][plugin:rollup@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.706] [error][status][plugin:remote_clusters@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.707] [error][status][plugin:cross_cluster_replication@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.707] [error][status][plugin:file_upload@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.708] [error][status][plugin:snapshot_restore@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.709] [error][status][plugin:reporting@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
  log   [17:18:30.710] [error][status][plugin:security@7.4.0] Status changed from yellow to red - Request Timeout after 30000ms
 error  [17:18:30.714] [warning][process] UnhandledPromiseRejectionWarning: Error: Request Timeout after 30000ms
    at C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:397:9
    at Timeout.<anonymous> (C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:429:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
    at emitWarning (internal/process/promises.js:81:15)
    at emitPromiseRejectionWarnings (internal/process/promises.js:120:9)
    at process._tickCallback (internal/process/next_tick.js:69:34)
 error  [17:18:30.716] [warning][process] Error: Request Timeout after 30000ms
    at C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:397:9
    at Timeout.<anonymous> (C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:429:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
  log   [17:18:31.502] [fatal][root] { Error: Request Timeout after 30000ms
    at C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:397:9
    at Timeout.<anonymous> (C:\kibana-7.4.0-windows-x86_64\node_modules\elasticsearch\src\lib\transport.js:429:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
  status: undefined,
  displayName: 'RequestTimeout',
  message: 'Request Timeout after 30000ms',
  body: undefined,
  isBoom: true,
  isServer: true,
  data: null,
  output:
   { statusCode: 503,
     payload:
      { statusCode: 503,
        error: 'Service Unavailable',
        message: 'Request Timeout after 30000ms' },
     headers: {} },
  reformat: [Function],
  [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }
  log   [17:18:31.510] [info][plugins-system] Stopping all plugins.
  log   [17:18:31.511] [info][data][plugins] Stopping plugin
  log   [17:18:31.512] [info][plugins][translations] Stopping plugin
  log   [17:18:31.513] [info][plugins][security] Stopping plugin

 FATAL  Error: Request Timeout after 30000ms
6

By the way,

When Kibana was issuing/logging the errors, the elasticsearch console did not show any activity.

7

I found an unusual way to cause Kibana to have these timeouts. If I start elasticsearch.bat in a cmd window and select any text in that window (even just clicking on a single character) that blocks Elasticsearch from logging anything and blocks the whole process. From that point on, Kibana will hit timeouts. Is it possible your cmd window is blocking Elasticsearch ?

The exact timeouts you see logged would depend on where Kibana was in the startup process when Elasticsearch got blocked.

Lee

7 Likes
8

Hi Lee,
yes, you are right, it seems that the command prompt window is very sensitive (elastic search command window)

I restarted it and now it works, meaning kibana is accessible and no errors.

since elastic search through the command prompt is very sensitive, is it possible to run elastic search as a windows service? what about Kibana?

thanks,
Francisco.

9

Elasticsearch does have a script to create a service on Windows (and also an msi installer which might create the service automatically). Instructions here;
https://www.elastic.co/guide/en/elasticsearch/reference/current/zip-windows.html

Kibana does not have such a feature yet. There are future plans for it. But there are some ways to do it. Here's a page about NSSM; https://imsavva.com/how-to-install-kibana-as-a-windows-service/ I used this NSSM approach on some automated tests.

Or you might be able to start Kibana in the background like this (I haven't tried this yet);
START /B kibana.bat

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.