# Can't turn on monitoring
**URL:** https://discuss.elastic.co/t/cant-turn-on-monitoring/290356
**Category:** Kibana
**Tags:** elastic-stack-monitoring
**Created:** [November 28, 2021, 8:41am UTC](https://discuss.elastic.co/t/cant-turn-on-monitoring/290356 "2021-11-28T08:41:55Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@avnere](https://discuss.elastic.co/u/avnere)
#### Post date: [November 28, 2021, 8:41am UTC](https://discuss.elastic.co/t/cant-turn-on-monitoring/290356/1 "2021-11-28T08:41:55Z")
Hi,
I have installed Kibana for the new cluster.
When going to Stack Monitoring. It shows “We couldn't activate monitoring”
In addition log prints “Unable to bulk upload the stats payload to the local cluster”
Dev tools and Index management are working properly.
I tried the following:
- Checked kibana status ([http://10.0.59.155:5601/status](http://10.0.59.155:5601/status)) --\> Seems OK.
- Checked if .monitoring index exist --\> does not exist.
- Checked if monitoring.collection.enabled=true --\> true
- Restarted the nodes and kibana.
Is it familiar to you?
Can you assist on it?
Thanks,
Avner
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 28, 2021, 9:29pm UTC](https://discuss.elastic.co/t/cant-turn-on-monitoring/290356/2 "2021-11-28T21:29:54Z")
Welcome to our community! 😃
> [@avnere](#):
>
> In addition log prints “Unable to bulk upload the stats payload to the local cluster”
Can you please post the entire Kibana and Elasticsearch log for us?
---
### Author:  [@avnere](https://discuss.elastic.co/u/avnere)
#### Post date: [November 29, 2021, 12:47pm UTC](https://discuss.elastic.co/t/cant-turn-on-monitoring/290356/3 "2021-11-29T12:47:28Z")
Kibana Log:
> Last login: Thu Nov 25 17:49:02 2021 from ae-dev-win10.ad.shared
> [xservice@iky-ed-lx-st05 ~]$ cd /storage/elastic/kibana/kibana-7.13.1-linux-x86\_64/bin/
> [xservice@iky-ed-lx-st05 bin]$ ./kibana
> log [17:58:27.673] [info][plugins-service] Plugin "timelines" is disabled.
> log [17:58:27.753] [warning][config][deprecation] plugins.scanDirs is deprecated and is no longer used
> log [17:58:27.753] [warning][config][deprecation] Config key [monitoring.cluster\_alerts.email\_notifications.email\_address] will be required for email notifications to work in 8.0."
> log [17:58:27.961] [info][plugins-system] Setting up [106] plugins: [taskManager,licensing,globalSearch,globalSearchProviders,banners,code,usageCollection,xpackLegacy,telemetryCollectionManager,telemetry,telemetryCollectionXpack,kibanaUsageCollection,securityOss,share,newsfeed,mapsEms,mapsLegacy,kibanaLegacy,translations,licenseApiGuard,legacyExport,embeddable,uiActionsEnhanced,esUiShared,expressions,charts,bfetch,data,home,apmOss,console,consoleExtensions,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,savedObjects,visualizations,visTypeMarkdown,visTypeVislib,visTypeMetric,visTypeVega,visTypeTimelion,features,licenseManagement,watcher,visTypeTable,visTypeTagcloud,visTypeXy,tileMap,regionMap,presentationUtil,canvas,graph,timelion,dashboard,dashboardEnhanced,visualize,visTypeTimeseries,inputControlVis,indexPatternManagement,discover,discoverEnhanced,savedObjectsManagement,spaces,security,savedObjectsTagging,lens,reporting,lists,encryptedSavedObjects,dataEnhanced,dashboardMode,cloud,upgradeAssistant,snapshotRestore,enterpriseSearch,fleet,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,beatsManagement,transform,ingestPipelines,fileUpload,maps,fileDataVisualizer,eventLog,actions,alerting,triggersActionsUi,stackAlerts,ruleRegistry,observability,osquery,ml,securitySolution,cases,infra,monitoring,logstash,apm,uptime]
> log [17:58:27.963] [info][plugins][taskManager] TaskManager is identified by the Kibana UUID: 05a69651-9eeb-4aaf-96a3-7925f6244ccb
> log [17:58:28.172] [warning][config][plugins][security] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
> log [17:58:28.172] [warning][config][plugins][security] Session cookies will be transmitted over insecure connections. This is not recommended.
> log [17:58:28.210] [warning][config][plugins][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
> log [17:58:28.216] [warning][config][plugins][reporting] Chromium sandbox provides an additional layer of protection, but is not supported for Linux CentOS 7.9.2009 OS. Automatically setting 'xpack.reporting.capture.browser.chromium.disableSandbox: true'.
> log [17:58:28.217] [warning][encryptedSavedObjects][plugins] Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
> log [17:58:28.316] [warning][actions][actions][plugins] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
> log [17:58:28.328] [warning][alerting][alerting][plugins][plugins] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
> log [17:58:28.393] [info][monitoring][monitoring][plugins] config sourced from: production cluster
> log [17:58:28.597] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
> log [17:58:28.659] [info][savedobjects-service] Starting saved objects migrations
> log [17:58:28.699] [info][savedobjects-service] [.kibana\_task\_manager] INIT -\> OUTDATED\_DOCUMENTS\_SEARCH. took: 10ms.
> log [17:58:28.715] [info][savedobjects-service] [.kibana] INIT -\> OUTDATED\_DOCUMENTS\_SEARCH. took: 28ms.
> log [17:58:28.720] [info][savedobjects-service] [.kibana\_task\_manager] OUTDATED\_DOCUMENTS\_SEARCH -\> UPDATE\_TARGET\_MAPPINGS. took: 21ms.
> log [17:58:28.730] [info][savedobjects-service] [.kibana] OUTDATED\_DOCUMENTS\_SEARCH -\> UPDATE\_TARGET\_MAPPINGS. took: 15ms.
> log [17:58:28.737] [info][savedobjects-service] [.kibana\_task\_manager] UPDATE\_TARGET\_MAPPINGS -\> UPDATE\_TARGET\_MAPPINGS\_WAIT\_FOR\_TASK. took: 17ms.
> log [17:58:28.788] [info][savedobjects-service] [.kibana] UPDATE\_TARGET\_MAPPINGS -\> UPDATE\_TARGET\_MAPPINGS\_WAIT\_FOR\_TASK. took: 58ms.
> log [17:58:29.611] [info][savedobjects-service] [.kibana] UPDATE\_TARGET\_MAPPINGS\_WAIT\_FOR\_TASK -\> DONE. took: 823ms.
> log [17:58:29.611] [info][savedobjects-service] [.kibana] Migration completed after 924ms
> log [17:58:29.646] [info][savedobjects-service] [.kibana\_task\_manager] UPDATE\_TARGET\_MAPPINGS\_WAIT\_FOR\_TASK -\> DONE. took: 909ms.
> log [17:58:29.646] [info][savedobjects-service] [.kibana\_task\_manager] Migration completed after 957ms
> log [17:58:29.711] [info][plugins-system] Starting [106] plugins: [taskManager,licensing,globalSearch,globalSearchProviders,banners,code,usageCollection,xpackLegacy,telemetryCollectionManager,telemetry,telemetryCollectionXpack,kibanaUsageCollection,securityOss,share,newsfeed,mapsEms,mapsLegacy,kibanaLegacy,translations,licenseApiGuard,legacyExport,embeddable,uiActionsEnhanced,esUiShared,expressions,charts,bfetch,data,home,apmOss,console,consoleExtensions,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,savedObjects,visualizations,visTypeMarkdown,visTypeVislib,visTypeMetric,visTypeVega,visTypeTimelion,features,licenseManagement,watcher,visTypeTable,visTypeTagcloud,visTypeXy,tileMap,regionMap,presentationUtil,canvas,graph,timelion,dashboard,dashboardEnhanced,visualize,visTypeTimeseries,inputControlVis,indexPatternManagement,discover,discoverEnhanced,savedObjectsManagement,spaces,security,savedObjectsTagging,lens,reporting,lists,encryptedSavedObjects,dataEnhanced,dashboardMode,cloud,upgradeAssistant,snapshotRestore,enterpriseSearch,fleet,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,beatsManagement,transform,ingestPipelines,fileUpload,maps,fileDataVisualizer,eventLog,actions,alerting,triggersActionsUi,stackAlerts,ruleRegistry,observability,osquery,ml,securitySolution,cases,infra,monitoring,logstash,apm,uptime]
> log [17:58:32.312] [info][server][Kibana][http] http server running at [http://10.0.59.155:5601](http://10.0.59.155:5601)
> log [17:58:32.376] [info][kibana-monitoring][monitoring][monitoring][plugins] Starting monitoring stats collection
> log [17:58:32.391] [warning][kibana-monitoring][monitoring][monitoring][plugins] ResponseError: export\_exception
> at onBody (/opt/xplorer/elastic/kibana/kibana-7.13.1-linux-x86\_64/node\_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
> at IncomingMessage.onEnd (/opt/xplorer/elastic/kibana/kibana-7.13.1-linux-x86\_64/node\_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
> at IncomingMessage.emit (events.js:327:22)
> at endReadableNT (internal/streams/readable.js:1327:12)
> at processTicksAndRejections (internal/process/task\_queues.js:80:21)
> log [17:58:32.391] [warning][kibana-monitoring][monitoring][monitoring][plugins] Unable to bulk upload the stats payload to the local cluster
> log [17:58:32.397] [info][plugins][securitySolution] Dependent plugin setup complete - Starting ManifestTask
> log [17:58:32.976] [info][plugins][reporting] Browser executable: /opt/xplorer/elastic/kibana/kibana-7.13.1-linux-x86\_64/x-pack/plugins/reporting/chromium/headless\_shell-linux\_x64/headless\_shell
> log [17:58:32.976] [warning][plugins][reporting] Enabling the Chromium sandbox provides an additional layer of protection.
> log [17:58:42.387] [warning][kibana-monitoring][monitoring][monitoring][plugins] ResponseError: export\_exception
> at onBody (/opt/xplorer/elastic/kibana/kibana-7.13.1-linux-x86\_64/node\_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
> at IncomingMessage.onEnd (/opt/xplorer/elastic/kibana/kibana-7.13.1-linux-x86\_64/node\_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
> at IncomingMessage.emit (events.js:327:22)
> at endReadableNT (internal/streams/readable.js:1327:12)
> at processTicksAndRejections (internal/process/task\_queues.js:80:21)
> log [17:58:42.388] [warning][kibana-monitoring][monitoring][monitoring][plugins] Unable to bulk upload the stats payload to the local cluster
> log [17:58:52.387] [warning][kibana-monitoring][monitoring][monitoring][plugins] ResponseError: export\_exception
> at onBody (/opt/xplorer/elastic/kibana/kibana-7.13.1-linux-x86\_64/node\_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
> at IncomingMessage.onEnd (/opt/xplorer/elastic/kibana/kibana-7.13.1-linux-x86\_64/node\_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
> at IncomingMessage.emit (events.js:327:22)
> at endReadableNT (internal/streams/readable.js:1327:12)
> at processTicksAndRejections (internal/process/task\_queues.js:80:21)
> log [17:58:52.387] [warning][kibana-monitoring][monitoring][monitoring][plugins] Unable to bulk upload the stats payload to the local cluster
---
### Author:  [@avnere](https://discuss.elastic.co/u/avnere)
#### Post date: [November 29, 2021, 12:59pm UTC](https://discuss.elastic.co/t/cant-turn-on-monitoring/290356/4 "2021-11-29T12:59:30Z")
elastic Log:
> {"type": "server", "timestamp": "2021-11-29T00:00:09,846+02:00", "level": "WARN", "component": "o.e.x.m.MonitoringService", "cluster.name": "elastic-tests-lab162", "node.name": "masterClient-node-1", "message": "monitoring execution failed", "cluster.uuid": "famMKqRTQzmTJbdsxh5ZqA", "node.id": "kYbCqL1dTOK-Vt70n7JFvg" ,
> "stacktrace": ["org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulks",
> "at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$0(ExportBulk.java:110) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:142) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.lambda$doFlush$1(LocalBulk.java:113) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:142) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:38) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:92) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.ActionListener$Delegating.onFailure(ActionListener.java:66) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onFailure(ActionListener.java:397) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.bulk.TransportBulkAction.doInternalExecute(TransportBulkAction.java:211) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:162) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:89) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:173) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:42) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:171) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:149) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:77) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:86) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:66) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:402) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.java:484) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:119) [x-pack-core-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.doFlush(LocalBulk.java:106) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.flush(ExportBulk.java:64) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$1(ExportBulk.java:108) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:103) [x-pack-core-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.doFlush(ExportBulk.java:124) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.flush(ExportBulk.java:64) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.Exporters.doExport(Exporters.java:284) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.Exporters.lambda$export$3(Exporters.java:259) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:134) [elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.delegateIfComplete(Exporters.java:364) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.onResponse(Exporters.java:343) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.onResponse(Exporters.java:314) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.local.LocalExporter.openBulk(LocalExporter.java:243) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.Exporters.wrapExportBulk(Exporters.java:245) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.exporter.Exporters.export(Exporters.java:257) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:262) [x-pack-monitoring-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.13.1.jar:7.13.1]",
> "at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]",
> "at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]",
> "at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673) [elasticsearch-7.13.1.jar:7.13.1]",
> "at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]",
> "at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]",
> "at java.lang.Thread.run(Thread.java:831) [?:?]",
> "Caused by: org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulk [default\_local]",
> "... 42 more",
> "Caused by: java.lang.IllegalStateException: There are no ingest nodes in this cluster, unable to forward request to an ingest node.",
> "at org.elasticsearch.action.ingest.IngestActionForwarder.randomIngestNode(IngestActionForwarder.java:47) ~[elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.ingest.IngestActionForwarder.forwardIngestRequest(IngestActionForwarder.java:40) ~[elasticsearch-7.13.1.jar:7.13.1]",
> "at org.elasticsearch.action.bulk.TransportBulkAction.doInternalExecute(TransportBulkAction.java:208) ~[elasticsearch-7.13.1.jar:7.13.1]",
> "... 35 more"] }
---
### Author:  [@avnere](https://discuss.elastic.co/u/avnere)
#### Post date: [December 1, 2021, 11:42am UTC](https://discuss.elastic.co/t/cant-turn-on-monitoring/290356/5 "2021-12-01T11:42:01Z")
Can you help with it?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 29, 2021, 11:42am UTC](https://discuss.elastic.co/t/cant-turn-on-monitoring/290356/6 "2021-12-29T11:42:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.