I'm running kibana 7.14.2 on Linux from my localhost and with wazuh installed, however the server keep crashing after some time functioning properly. I have tried different approaches including limit the memory use from Elasticsearch and etc, please someone help me look into this. Thanks!!!!
Welcome to our community! 
Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them 
What does your Kibana log show?
Hello!
Thanks for your reply, my log said I have failed to download chrome-win.zip and I have tried to open the line to see what that does but showed 404. That is the error message, after that server just crashed. Do you need the full log?
Appreciate your help!
It'd be useful if you could post the logs, yes.
Here is the log, thanks!
Error: Failed to download https://commondatastorage.googleapis.com/chromium-browser-snapshots/Win/856583/chrome-win.zip
at /usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:55:31
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async Promise.all (index 3)
at /usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:37:7
at async Promise.all (index 0)
at ensureDownloaded (/usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:32:3)
at ensureBrowserDownloaded (/usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:21:3)
at backgroundInstall (/usr/local/kibana/x-pack/plugins/reporting/server/browsers/install.ts:42:7)
Terminating process...
server crashed with status code 1
Is there more to the log?
server log [11:25:38.442] [debug][kibana-monitoring][monitoring][monitoring][plugins] Uploaded bulk stats payload to the local cluster
server log [11:25:39.892] [debug][Elasticsearch][query][taskManager] 200
POST /.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true&conflicts=proceed
{"query":{"bool":{"must":[{"term":{"type":"task"}},{"bool":{"must":[{"bool":{"should":[{"bool":{"must":[{"term":{"task.status":"idle"}},{"range":{"task.runAt":{"lte":"now"}}}]}},{"bool":{"must":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}]}},{"range":{"task.retryAt":{"lte":"now"}}}]}}]}}],"filter":[{"bool":{"must_not":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}],"must":{"range":{"task.retryAt":{"gt":"now"}}}}}]}}]}}]}},"script":{"source":"\n if (params.claimableTaskTypes.contains(ctx._source.task.taskType)) {\n if (ctx._source.task.schedule != null || ctx._source.task.attempts < params.taskMaxAttempts[ctx._source.task.taskType] || params.claimTasksById.contains(ctx._id)) {\n ctx._source.task.status = "claiming"; ctx._source.task.ownerId=params.fieldUpdates.ownerId; ctx._source.task.retryAt=params.fieldUpdates.retryAt;\n } else {\n ctx._source.task.status = "failed";\n }\n } else if (params.skippedTaskTypes.contains(ctx._source.task.taskType) && params.claimTasksById.contains(ctx._id)) {\n ctx._source.task.status = "claiming"; ctx._source.task.ownerId=params.fieldUpdates.ownerId; ctx._source.task.retryAt=params.fieldUpdates.retryAt;\n } else if (!params.skippedTaskTypes.contains(ctx._source.task.taskType)) {\n ctx._source.task.status = "unrecognized";\n } else {\n ctx.op = "noop";\n }","lang":"painless","params":{"fieldUpdates":{"ownerId":"kibana:1fd4fe8e-b022-4ed4-9835-3869b933f999","retryAt":"2022-04-26T03:26:09.887Z"},"claimTasksById":,"claimableTaskTypes":["report:execute"],"skippedTaskTypes":["session_cleanup","lens_telemetry","reports:monitor","search_sessions_monitor","search_sessions_cleanup","search_sessions_expire","actions:.email","actions:.index","actions:.pagerduty","actions:.swimlane","actions:.server-log","actions:.slack","actions:.webhook","actions:.servicenow","actions:.servicenow-sir","actions:.jira","actions:.resilient","actions:.teams","actions_telemetry","cleanup_failed_action_executions","alerting_telemetry","alerts_invalidate_api_keys","alerting_health_check","alerting:.index-threshold","alerting:.geo-containment","alerting:.es-query","alerting:xpack.ml.anomaly_detection_alert","alerting:siem.signals","alerting:siem.notifications","endpoint:user-artifact-packager","security:endpoint-diagnostics","security:endpoint-meta-telemetry","alerting:xpack.uptime.alerts.monitorStatus","alerting:xpack.uptime.alerts.tlsCertificate","alerting:xpack.uptime.alerts.tls","alerting:xpack.uptime.alerts.durationAnomaly","alerting:metrics.alert.threshold","alerting:metrics.alert.inventory.threshold","alerting:metrics.alert.anomaly","alerting:logs.alert.document.count","alerting:monitoring_alert_cluster_health","alerting:monitoring_alert_license_expiration","alerting:monitoring_alert_cpu_usage","alerting:monitoring_alert_missing_monitoring_data","alerting:monitoring_alert_disk_usage","alerting:monitoring_alert_thread_pool_search_rejections","alerting:monitoring_alert_thread_pool_write_rejections","alerting:monitoring_alert_jvm_memory_usage","alerting:monitoring_alert_nodes_changed","alerting:monitoring_alert_logstash_version_mismatch","alerting:monitoring_alert_kibana_version_mismatch","alerting:monitoring_alert_elasticsearch_version_mismatch","alerting:monitoring_ccr_read_exceptions","alerting:monitoring_shard_size","apm-telemetry-task","alerting:apm.transaction_duration","alerting:apm.transaction_duration_anomaly","alerting:apm.error_rate","alerting:apm.transaction_error_rate"],"taskMaxAttempts":{"report:execute":1}}},"sort":[{"_script":{"type":"number","order":"asc","script":{"lang":"painless","source":"\nif (doc['task.retryAt'].size()!=0) {\n return doc['task.retryAt'].value.toInstant().toEpochMilli();\n}\nif (doc['task.runAt'].size()!=0) {\n return doc['task.runAt'].value.toInstant().toEpochMilli();\n}\n "}}}],"max_docs":1}
server log [11:25:39.898] [debug][Elasticsearch][query][taskManager] 200
POST /.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true&conflicts=proceed
{"query":{"bool":{"must":[{"term":{"type":"task"}},{"bool":{"must":[{"bool":{"should":[{"bool":{"must":[{"term":{"task.status":"idle"}},{"range":{"task.runAt":{"lte":"now"}}}]}},{"bool":{"must":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}]}},{"range":{"task.retryAt":{"lte":"now"}}}]}}]}}],"filter":[{"bool":{"must_not":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}],"must":{"range":{"task.retryAt":{"gt":"now"}}}}}]}}]}}]}},"script":{"source":"\n if (params.claimableTaskTypes.contains(ctx._source.task.taskType)) {\n if (ctx._source.task.schedule != null || ctx._source.task.attempts < params.taskMaxAttempts[ctx._source.task.taskType] || params.claimTasksById.contains(ctx._id)) {\n ctx._source.task.status = "claiming"; ctx._source.task.ownerId=params.fieldUpdates.ownerId; ctx._source.task.retryAt=params.fieldUpdates.retryAt;\n } else {\n ctx._source.task.status = "failed";\n }\n } else if (params.skippedTaskTypes.contains(ctx._source.task.taskType) && params.claimTasksById.contains(ctx._id)) {\n ctx._source.task.status = "claiming"; ctx._source.task.ownerId=params.fieldUpdates.ownerId; ctx._source.task.retryAt=params.fieldUpdates.retryAt;\n } else if (!params.skippedTaskTypes.contains(ctx._source.task.taskType)) {\n ctx._source.task.status = "unrecognized";\n } else {\n ctx.op = "noop";\n }","lang":"painless","params":{"fieldUpdates":{"ownerId":"kibana:1fd4fe8e-b022-4ed4-9835-3869b933f999","retryAt":"2022-04-26T03:26:09.887Z"},"claimTasksById":,"claimableTaskTypes":["session_cleanup","lens_telemetry","reports:monitor","search_sessions_monitor","search_sessions_cleanup","search_sessions_expire","actions:.email","actions:.index","actions:.pagerduty","actions:.swimlane","actions:.server-log","actions:.slack","actions:.webhook","actions:.servicenow","actions:.servicenow-sir","actions:.jira","actions:.resilient","actions:.teams","actions_telemetry","cleanup_failed_action_executions","alerting_telemetry","alerts_invalidate_api_keys","alerting_health_check","alerting:.index-threshold","alerting:.geo-containment","alerting:.es-query","alerting:xpack.ml.anomaly_detection_alert","alerting:siem.signals","alerting:siem.notifications","endpoint:user-artifact-packager","security:endpoint-diagnostics","security:endpoint-meta-telemetry","alerting:xpack.uptime.alerts.monitorStatus","alerting:xpack.uptime.alerts.tlsCertificate","alerting:xpack.uptime.alerts.tls","alerting:xpack.uptime.alerts.durationAnomaly","alerting:metrics.alert.threshold","alerting:metrics.alert.inventory.threshold","alerting:metrics.alert.anomaly","alerting:logs.alert.document.count","alerting:monitoring_alert_cluster_health","alerting:monitoring_alert_license_expiration","alerting:monitoring_alert_cpu_usage","alerting:monitoring_alert_missing_monitoring_data","alerting:monitoring_alert_disk_usage","alerting:monitoring_alert_thread_pool_search_rejections","alerting:monitoring_alert_thread_pool_write_rejections","alerting:monitoring_alert_jvm_memory_usage","alerting:monitoring_alert_nodes_changed","alerting:monitoring_alert_logstash_version_mismatch","alerting:monitoring_alert_kibana_version_mismatch","alerting:monitoring_alert_elasticsearch_version_mismatch","alerting:monitoring_ccr_read_exceptions","alerting:monitoring_shard_size","apm-telemetry-task","alerting:apm.transaction_duration","alerting:apm.transaction_duration_anomaly","alerting:apm.error_rate","alerting:apm.transaction_error_rate"],"skippedTaskTypes":["report:execute"],"taskMaxAttempts":{"session_cleanup":3,"lens_telemetry":3,"reports:monitor":1,"search_sessions_monitor":3,"search_sessions_cleanup":3,"search_sessions_expire":3,"actions:.email":1,"actions:.index":1,"actions:.pagerduty":1,"actions:.swimlane":1,"actions:.server-log":1,"actions:.slack":1,"actions:.webhook":1,"actions:.servicenow":1,"actions:.servicenow-sir":1,"actions:.jira":1,"actions:.resilient":1,"actions:.teams":1,"actions_telemetry":3,"cleanup_failed_action_executions":3,"alerting_telemetry":3,"alerts_invalidate_api_keys":3,"alerting_health_check":3,"alerting:.index-threshold":3,"alerting:.geo-containment":3,"alerting:.es-query":3,"alerting:xpack.ml.anomaly_detection_alert":3,"alerting:siem.signals":3,"alerting:siem.notifications":3,"endpoint:user-artifact-packager":3,"security:endpoint-diagnostics":3,"security:endpoint-meta-telemetry":3,"alerting:xpack.uptime.alerts.monitorStatus":3,"alerting:xpack.uptime.alerts.tlsCertificate":3,"alerting:xpack.uptime.alerts.tls":3,"alerting:xpack.uptime.alerts.durationAnomaly":3,"alerting:metrics.alert.threshold":3,"alerting:metrics.alert.inventory.threshold":3,"alerting:metrics.alert.anomaly":3,"alerting:logs.alert.document.count":3,"alerting:monitoring_alert_cluster_health":3,"alerting:monitoring_alert_license_expiration":3,"alerting:monitoring_alert_cpu_usage":3,"alerting:monitoring_alert_missing_monitoring_data":3,"alerting:monitoring_alert_disk_usage":3,"alerting:monitoring_alert_thread_pool_search_rejections":3,"alerting:monitoring_alert_thread_pool_write_rejections":3,"alerting:monitoring_alert_jvm_memory_usage":3,"alerting:monitoring_alert_nodes_changed":3,"alerting:monitoring_alert_logstash_version_mismatch":3,"alerting:monitoring_alert_kibana_version_mismatch":3,"alerting:monitoring_alert_elasticsearch_version_mismatch":3,"alerting:monitoring_ccr_read_exceptions":3,"alerting:monitoring_shard_size":3,"apm-telemetry-task":3,"alerting:apm.transaction_duration":3,"alerting:apm.transaction_duration_anomaly":3,"alerting:apm.error_rate":3,"alerting:apm.transaction_error_rate":3}}},"sort":[{"_script":{"type":"number","order":"asc","script":{"lang":"painless","source":"\nif (doc['task.retryAt'].size()!=0) {\n return doc['task.retryAt'].value.toInstant().toEpochMilli();\n}\nif (doc['task.runAt'].size()!=0) {\n return doc['task.runAt'].value.toInstant().toEpochMilli();\n}\n "}}}],"max_docs":10}
server log [11:25:40.283] [error][plugins][reporting] Error: connect ETIMEDOUT 243.185.187.39:443
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1148:16) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '243.185.187.39',
port: 443,
config: {
url: 'https://commondatastorage.googleapis.com/chromium-browser-snapshots/Win/856583/chrome-win.zip',
method: 'get',
headers: {
Accept: 'application/json, text/plain, /',
'User-Agent': 'axios/0.21.1'
},
transformRequest: [ [Function: transformRequest] ],
transformResponse: [ [Function: transformResponse] ],
timeout: 0,
adapter: [Function: httpAdapter],
responseType: 'stream',
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: -1,
maxBodyLength: -1,
validateStatus: [Function: validateStatus],
data: undefined
},
request: <ref *1> Writable {
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: false,
ending: false,
ended: false,
finished: false,
destroyed: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
afterWriteTickInfo: null,
buffered: ,
bufferedIndex: 0,
allBuffers: true,
allNoop: true,
pendingcb: 0,
prefinished: false,
errorEmitted: false,
emitClose: true,
autoDestroy: true,
errored: null,
closed: false
},
_events: [Object: null prototype] {
response: [Function: handleResponse],
error: [Function: handleRequestError]
},
_eventsCount: 2,
_maxListeners: undefined,
_options: {
maxRedirects: 21,
maxBodyLength: 10485760,
protocol: 'https:',
path: '/chromium-browser-snapshots/Win/856583/chrome-win.zip',
method: 'GET',
headers: [Object],
agent: undefined,
agents: [Object],
auth: undefined,
hostname: 'commondatastorage.googleapis.com',
port: null,
nativeProtocols: [Object],
pathname: '/chromium-browser-snapshots/Win/856583/chrome-win.zip'
},
_ended: true,
_ending: true,
_redirectCount: 0,
_redirects: ,
_requestBodyLength: 0,
_requestBodyBuffers: ,
_onNativeResponse: [Function (anonymous)],
_currentRequest: ClientRequest {
_events: [Object: null prototype],
_eventsCount: 7,
_maxListeners: undefined,
outputData: ,
outputSize: 0,
writable: true,
destroyed: false,
_last: true,
chunkedEncoding: false,
shouldKeepAlive: false,
_defaultKeepAlive: true,
useChunkedEncodingByDefault: false,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
_contentLength: 0,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
socket: [TLSSocket],
_header: 'GET /chromium-browser-snapshots/Win/856583/chrome-win.zip HTTP/1.1\r\n' +
'Accept: application/json, text/plain, /\r\n' +
'User-Agent: axios/0.21.1\r\n' +
'Host: commondatastorage.googleapis.com\r\n' +
'Connection: close\r\n' +
'\r\n',
_keepAliveTimeout: 0,
_onPendingData: [Function: noopPendingOutput],
agent: [Agent],
socketPath: undefined,
method: 'GET',
maxHeaderSize: undefined,
insecureHTTPParser: undefined,
path: '/chromium-browser-snapshots/Win/856583/chrome-win.zip',
_ended: false,
res: null,
aborted: false,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
reusedSocket: false,
host: 'commondatastorage.googleapis.com',
protocol: 'https:',
_redirectable: [Circular *1],
[Symbol(kCapture)]: false,
[Symbol(kNeedDrain)]: false,
[Symbol(corked)]: 0,
[Symbol(kOutHeaders)]: [Object: null prototype]
},
_currentUrl: 'https://commondatastorage.googleapis.com/chromium-browser-snapshots/Win/856583/chrome-win.zip',
[Symbol(kCapture)]: false
},
response: undefined,
isAxiosError: true,
toJSON: [Function: toJSON]
}
Unhandled Promise rejection detected:
Error: Failed to download https://commondatastorage.googleapis.com/chromium-browser-snapshots/Win/856583/chrome-win.zip
at /usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:55:31
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async Promise.all (index 3)
at /usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:37:7
at async Promise.all (index 0)
at ensureDownloaded (/usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:32:3)
at ensureBrowserDownloaded (/usr/local/kibana/x-pack/plugins/reporting/server/browsers/download/ensure_downloaded.ts:21:3)
at backgroundInstall (/usr/local/kibana/x-pack/plugins/reporting/server/browsers/install.ts:42:7)
Terminating process...
server crashed with status code 1
Here's more of that, thanks!
Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you
Where is this IP coming from?
I think that is the address which the system is trying to download the chrome-win.zip
Can you do a dig or similar on your Kibana host to see if that is what commondatastorage.googleapis.com resolves to?
I have tried, it shows 404.
I stopped the crashing by configuring my kibana.yml files, changing 'localhost' to my IP address. Thanks for the help!!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.