I deployed ELK stack with kubernetes, it works with no error if i use one replica for elasticsearch deployment .
But when it increase a replicas to 2 i get the following error
[2023-11-27T18:03:54.296+00:00][INFO ][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
[2023-11-27T18:03:54.296+00:00][INFO ][savedobjects-service] Starting saved objects migrations
[2023-11-27T18:03:54.319+00:00][WARN ][savedobjects-service] Unable to connect to Elasticsearch. Error: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana]
[2023-11-27T18:03:54.453+00:00][INFO ][savedobjects-service] [.kibana_security_solution] INIT -> CREATE_NEW_TARGET. took: 113ms.
[2023-11-27T18:03:54.487+00:00][INFO ][savedobjects-service] [.kibana_task_manager] INIT -> CREATE_NEW_TARGET. took: 158ms.
[2023-11-27T18:03:54.501+00:00][INFO ][savedobjects-service] [.kibana_alerting_cases] INIT -> CREATE_NEW_TARGET. took: 154ms.
[2023-11-27T18:03:54.524+00:00][INFO ][savedobjects-service] [.kibana] INIT -> CREATE_NEW_TARGET. took: 198ms.
[2023-11-27T18:03:54.529+00:00][INFO ][savedobjects-service] [.kibana_ingest] INIT -> CREATE_NEW_TARGET. took: 179ms.
[2023-11-27T18:03:54.587+00:00][INFO ][savedobjects-service] [.kibana_analytics] INIT -> CREATE_NEW_TARGET. took: 253ms.
[2023-11-27T18:03:55.246+00:00][INFO ][savedobjects-service] [.kibana] CREATE_NEW_TARGET -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 722ms.
[2023-11-27T18:03:55.247+00:00][INFO ][savedobjects-service] [.kibana] CHECK_VERSION_INDEX_READY_ACTIONS -> MARK_VERSION_INDEX_READY. took: 1ms.
[2023-11-27T18:03:55.261+00:00][INFO ][savedobjects-service] [.kibana_security_solution] CREATE_NEW_TARGET -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 808ms.
[2023-11-27T18:03:55.262+00:00][INFO ][savedobjects-service] [.kibana_security_solution] CHECK_VERSION_INDEX_READY_ACTIONS -> MARK_VERSION_INDEX_READY. took: 1ms.
[2023-11-27T18:03:55.265+00:00][INFO ][savedobjects-service] [.kibana_task_manager] CREATE_NEW_TARGET -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 778ms.
[2023-11-27T18:03:55.265+00:00][INFO ][savedobjects-service] [.kibana_task_manager] CHECK_VERSION_INDEX_READY_ACTIONS -> MARK_VERSION_INDEX_READY. took: 0ms.
[2023-11-27T18:03:55.270+00:00][INFO ][savedobjects-service] [.kibana_alerting_cases] CREATE_NEW_TARGET -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 770ms.
[2023-11-27T18:03:55.270+00:00][INFO ][savedobjects-service] [.kibana_alerting_cases] CHECK_VERSION_INDEX_READY_ACTIONS -> MARK_VERSION_INDEX_READY. took: 0ms.
[2023-11-27T18:03:55.294+00:00][INFO ][savedobjects-service] [.kibana_analytics] CREATE_NEW_TARGET -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 707ms.
[2023-11-27T18:03:55.295+00:00][INFO ][savedobjects-service] [.kibana_analytics] CHECK_VERSION_INDEX_READY_ACTIONS -> MARK_VERSION_INDEX_READY. took: 1ms.
[2023-11-27T18:03:55.359+00:00][INFO ][savedobjects-service] [.kibana_analytics] MARK_VERSION_INDEX_READY -> DONE. took: 64ms.
[2023-11-27T18:03:55.360+00:00][INFO ][savedobjects-service] [.kibana_analytics] Migration completed after 1026ms
[2023-11-27T18:03:55.381+00:00][INFO ][savedobjects-service] [.kibana] MARK_VERSION_INDEX_READY -> DONE. took: 134ms.
[2023-11-27T18:03:55.382+00:00][INFO ][savedobjects-service] [.kibana] Migration completed after 1056ms
[2023-11-27T18:03:55.384+00:00][INFO ][savedobjects-service] [.kibana_security_solution] MARK_VERSION_INDEX_READY -> DONE. took: 122ms.
[2023-11-27T18:03:55.384+00:00][INFO ][savedobjects-service] [.kibana_security_solution] Migration completed after 1044ms
[2023-11-27T18:03:55.386+00:00][INFO ][savedobjects-service] [.kibana_task_manager] MARK_VERSION_INDEX_READY -> DONE. took: 120ms.
[2023-11-27T18:03:55.386+00:00][INFO ][savedobjects-service] [.kibana_task_manager] Migration completed after 1057ms
[2023-11-27T18:03:55.389+00:00][INFO ][savedobjects-service] [.kibana_alerting_cases] MARK_VERSION_INDEX_READY -> DONE. took: 119ms.
[2023-11-27T18:03:55.389+00:00][INFO ][savedobjects-service] [.kibana_alerting_cases] Migration completed after 1043ms
[2023-11-27T18:03:55.432+00:00][INFO ][savedobjects-service] [.kibana_ingest] CREATE_NEW_TARGET -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 903ms.
[2023-11-27T18:03:55.433+00:00][INFO ][savedobjects-service] [.kibana_ingest] CHECK_VERSION_INDEX_READY_ACTIONS -> MARK_VERSION_INDEX_READY. took: 1ms.
[2023-11-27T18:03:55.469+00:00][INFO ][savedobjects-service] [.kibana_ingest] MARK_VERSION_INDEX_READY -> DONE. took: 36ms.
[2023-11-27T18:03:55.470+00:00][INFO ][savedobjects-service] [.kibana_ingest] Migration completed after 1120ms
[2023-11-27T18:03:55.483+00:00][INFO ][plugins-system.preboot] Stopping all plugins.
[2023-11-27T18:03:55.485+00:00][INFO ][plugins-system.preboot] All plugins stopped.
[2023-11-27T18:03:55.488+00:00][INFO ][plugins-system.standard] Starting [147] plugins: [devTools,translations,share,screenshotMode,usageCollection,telemetryCollectionManager,telemetryCollectionXpack,taskManager,kibanaUsageCollection,cloud,newsfeed,savedObjectsFinder,noDataPage,monitoringCollection,metricsDataAccess,licensing,mapsEms,globalSearch,globalSearchProviders,features,guidedOnboarding,banners,licenseApiGuard,customBranding,ftrApis,fieldFormats,expressions,screenshotting,esUiShared,customIntegrations,contentManagement,dataViews,home,searchprofiler,painlessLab,management,spaces,security,telemetry,licenseManagement,snapshotRestore,lists,files,encryptedSavedObjects,eventLog,actions,notifications,cloudDataMigration,advancedSettings,grokdebugger,console,bfetch,data,savedObjectsTagging,savedObjectsManagement,unifiedSearch,graph,alerting,embeddable,uiActionsEnhanced,savedSearch,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,fileUpload,ingestPipelines,ecsDataQualityDashboard,dataViewFieldEditor,dataViewManagement,charts,watcher,visualizations,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeMarkdown,visTypeHeatmap,inputControlVis,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,visTypeGauge,eventAnnotation,expressionXY,dashboard,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,links,lens,maps,discover,reporting,canvas,fleet,logExplorer,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,dataVisualizer,cases,timelines,sessionView,kubernetesSecurity,cloudDefend,threatIntelligence,osquery,observabilityLogExplorer,observabilityAIAssistant,logsShared,aiops,observability,observabilityOnboarding,ml,uptime,synthetics,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,elasticAssistant,securitySolution,securitySolutionEss,dashboardEnhanced,apmDataAccess,assetManager,apm]
[2023-11-27T18:03:56.319+00:00][INFO ][plugins.fleet] Task Fleet-Usage-Sender-1.1.3 scheduled with interval 1h
[2023-11-27T18:03:56.319+00:00][INFO ][plugins.fleet.fleet:check-deleted-files-task:1.0.1] Started with interval of [1d] and timeout of [2m]
[2023-11-27T18:03:56.340+00:00][INFO ][plugins.monitoring.monitoring] config sourced from: production cluster
[2023-11-27T18:03:56.404+00:00][INFO ][plugins.observability] Installing SLO shared resources
[2023-11-27T18:03:56.405+00:00][INFO ][plugins.observability] Installing SLO component template [.slo-observability.sli-mappings]
[2023-11-27T18:03:56.406+00:00][INFO ][plugins.observability] Installing SLO component template [.slo-observability.sli-settings]
[2023-11-27T18:03:56.406+00:00][INFO ][plugins.observability] Installing SLO component template [.slo-observability.summary-mappings]
[2023-11-27T18:03:56.407+00:00][INFO ][plugins.observability] Installing SLO component template [.slo-observability.summary-settings]
[2023-11-27T18:03:56.430+00:00][INFO ][plugins.alerting] Installing ILM policy .alerts-ilm-policy
[2023-11-27T18:03:56.433+00:00][INFO ][plugins.alerting] Installing component template .alerts-framework-mappings
[2023-11-27T18:03:56.435+00:00][INFO ][plugins.alerting] Installing component template .alerts-legacy-alert-mappings
[2023-11-27T18:03:56.470+00:00][INFO ][plugins.alerting] Installing component template .alerts-ecs-mappings
[2023-11-27T18:03:56.480+00:00][INFO ][plugins.ruleRegistry] Installing component template .alerts-technical-mappings
[2023-11-27T18:03:59.105+00:00][INFO ][http.server.Kibana] http server running at http://0.0.0.0:5601
[2023-11-27T18:03:59.203+00:00][INFO ][status] Kibana is now degraded
[2023-11-27T18:03:59.316+00:00][INFO ][plugins.telemetry] Telemetry collection is enabled. For more information on telemetry settings, refer to https://www.elastic.co/guide/en/kibana/8.11/telemetry-settings-kbn.html.
[2023-11-27T18:03:59.784+00:00][INFO ][plugins.monitoring.monitoring.kibana-monitoring] Starting monitoring stats collection
[2023-11-27T18:03:59.988+00:00][ERROR][plugins.osquery.telemetry_events] [task osquery:telemetry-packs:1.1.0]: error scheduling task, received Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
[2023-11-27T18:04:00.010+00:00][INFO ][plugins.ruleRegistry] Installing ILM policy .preview.alerts-security.alerts-policy
[2023-11-27T18:04:00.052+00:00][ERROR][plugins.alerting] Error scheduling Alerting-alerting_health_check, received Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
[2023-11-27T18:04:00.108+00:00][ERROR][plugins.securitySolution.telemetry_events] [task security:telemetry-detection-rules:1.0.0]: error scheduling task, received Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
[2023-11-27T18:04:00.116+00:00][ERROR][plugins.reporting] Error in Reporting start, reporting may not function properly
[2023-11-27T18:04:00.146+00:00][ERROR][plugins.reporting] ResponseError: Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:479:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
[2023-11-27T18:04:00.176+00:00][ERROR][plugins.securitySolution.telemetry_events] [task security:telemetry-prebuilt-rule-alerts:1.0.0]: error scheduling task, received Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
[2023-11-27T18:04:00.199+00:00][ERROR][plugins.securitySolution.telemetry_events] [task security:telemetry-timelines:1.0.0]: error scheduling task, received Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
[2023-11-27T18:04:00.209+00:00][ERROR][plugins.fleet] Error scheduling task, received error: ResponseError: Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
[2023-11-27T18:04:00.212+00:00][INFO ][plugins.fleet] Task Fleet-Usage-Logger-Task scheduled with interval 15m
[2023-11-27T18:04:00.255+00:00][INFO ][plugins.observability] Installing SLO index template [.slo-observability.sli]
[2023-11-27T18:04:00.264+00:00][INFO ][plugins.alerting] Installing component template .alerts-stack.alerts-mappings
[2023-11-27T18:04:00.265+00:00][INFO ][plugins.alerting] Installing component template .alerts-observability.slo.alerts-mappings
[2023-11-27T18:04:00.267+00:00][INFO ][plugins.alerting] Installing component template .alerts-observability.threshold.alerts-mappings
[2023-11-27T18:04:00.268+00:00][INFO ][plugins.alerting] Installing component template .alerts-ml.anomaly-detection.alerts-mappings
[2023-11-27T18:04:00.270+00:00][INFO ][plugins.alerting] Installing component template .alerts-observability.uptime.alerts-mappings
[2023-11-27T18:04:00.271+00:00][INFO ][plugins.alerting] Installing component template .alerts-observability.logs.alerts-mappings
[2023-11-27T18:04:00.271+00:00][INFO ][plugins.alerting] Installing component template .alerts-observability.metrics.alerts-mappings
[2023-11-27T18:04:00.278+00:00][INFO ][plugins.alerting] Installing component template .alerts-security.alerts-mappings
[2023-11-27T18:04:00.280+00:00][INFO ][plugins.alerting] Installing component template .alerts-observability.apm.alerts-mappings
[2023-11-27T18:04:00.323+00:00][WARN ][environment] Detected an unhandled Promise rejection: ResponseError: Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:479:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
at ClientTraced.CreateApi [as create] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/create.js:43:12)
[2023-11-27T18:04:00.325+00:00][WARN ][process] UnhandledPromiseRejectionWarning: ResponseError: Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:479:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
at ClientTraced.CreateApi [as create] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/create.js:43:12)
at emitUnhandledRejectionWarning (node:internal/process/promises:200:15)
at processPromiseRejections (node:internal/process/promises:279:11)
at processTicksAndRejections (node:internal/process/task_queues:96:32)
[2023-11-27T18:04:00.325+00:00][WARN ][process] ResponseError: Saved object index alias [.kibana_task_manager_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_task_manager_8.11.0] and [require_alias] request flag is [true] and [.kibana_task_manager_8.11.0] is not an alias
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:479:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
at ClientTraced.CreateApi [as create] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/create.js:43:12)
[2023-11-27T18:04:00.329+00:00][INFO ][plugins.ruleRegistry] Installing component template .preview.alerts-security.alerts-mappings
[2023-11-27T18:04:00.337+00:00][INFO ][plugins.observabilityAIAssistant.service] Creating concrete write index - .kibana-observability-ai-assistant-conversations-000001
[2023-11-27T18:04:00.340+00:00][ERROR][plugins.observability] Error installing resources shared for SLO: invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.slo-observability.sli] invalid, cause [index template [.slo-observability.sli] specifies component templates [.slo-observability.sli-mappings, .slo-observability.sli-settings] that do not exist]
[2023-11-27T18:04:00.340+00:00][ERROR][plugins.observability] Failed to install SLO common resources and summary transforms
[2023-11-27T18:04:00.751+00:00][INFO ][plugins.eventLog] Installing index template .kibana-event-log-template
[2023-11-27T18:04:00.783+00:00][INFO ][plugins.fleet] Agent policies updated by license change: []
[2023-11-27T18:04:00.785+00:00][INFO ][plugins.alerting] Installing index template .alerts-stack.alerts-default-index-template
[2023-11-27T18:04:00.789+00:00][INFO ][plugins.alerting] Installing index template .alerts-observability.logs.alerts-default-index-template
[2023-11-27T18:04:00.792+00:00][INFO ][plugins.alerting] Installing index template .alerts-observability.slo.alerts-default-index-template
[2023-11-27T18:04:00.794+00:00][INFO ][plugins.alerting] Installing index template .alerts-observability.apm.alerts-default-index-template
[2023-11-27T18:04:00.795+00:00][INFO ][plugins.alerting] Installing index template .alerts-observability.metrics.alerts-default-index-template
[2023-11-27T18:04:00.799+00:00][INFO ][plugins.alerting] Installing index template .alerts-observability.threshold.alerts-default-index-template
[2023-11-27T18:04:00.803+00:00][INFO ][plugins.alerting] Installing index template .alerts-security.alerts-default-index-template
[2023-11-27T18:04:00.808+00:00][INFO ][plugins.alerting] Installing index template .alerts-observability.uptime.alerts-default-index-template
[2023-11-27T18:04:00.810+00:00][INFO ][plugins.alerting] Installing index template .alerts-ml.anomaly-detection.alerts-default-index-template
[2023-11-27T18:04:00.824+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-stack.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-stack.alerts-default-index-template] invalid, cause [index template [.alerts-stack.alerts-default-index-template] specifies component templates [.alerts-framework-mappings] that do not exist]
[2023-11-27T18:04:00.824+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-stack.alerts-default-000001
[2023-11-27T18:04:00.826+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-observability.logs.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-observability.logs.alerts-default-index-template] invalid, cause [index template [.alerts-observability.logs.alerts-default-index-template] specifies component templates [.alerts-ecs-mappings, .alerts-legacy-alert-mappings] that do not exist]
[2023-11-27T18:04:00.829+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-observability.logs.alerts-default-000001
[2023-11-27T18:04:00.846+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-observability.slo.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-observability.slo.alerts-default-index-template] invalid, cause [index template [.alerts-observability.slo.alerts-default-index-template] specifies component templates [.alerts-legacy-alert-mappings] that do not exist]
[2023-11-27T18:04:00.847+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-observability.slo.alerts-default-000001
[2023-11-27T18:04:00.850+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-observability.apm.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-observability.apm.alerts-default-index-template] invalid, cause [index template [.alerts-observability.apm.alerts-default-index-template] specifies component templates [.alerts-framework-mappings] that do not exist]
[2023-11-27T18:04:00.850+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-observability.apm.alerts-default-000001
[2023-11-27T18:04:00.852+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-observability.metrics.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-observability.metrics.alerts-default-index-template] invalid, cause [index template [.alerts-observability.metrics.alerts-default-index-template] specifies component templates [.alerts-framework-mappings] that do not exist]
[2023-11-27T18:04:00.857+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-observability.metrics.alerts-default-000001
[2023-11-27T18:04:00.859+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-observability.threshold.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-observability.threshold.alerts-default-index-template] invalid, cause [index template [.alerts-observability.threshold.alerts-default-index-template] specifies component templates [.alerts-ecs-mappings] that do not exist]
[2023-11-27T18:04:00.862+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-observability.threshold.alerts-default-000001
[2023-11-27T18:04:00.863+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-security.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-security.alerts-default-index-template] invalid, cause [index template [.alerts-security.alerts-default-index-template] specifies component templates [.alerts-ecs-mappings, .alerts-legacy-alert-mappings] that do not exist]
[2023-11-27T18:04:00.864+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-security.alerts-default-000001
[2023-11-27T18:04:00.917+00:00][WARN ][plugins.usageCollection.usage-collection.usage-counters-service] ResponseError: Saved object index alias [.kibana_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_8.11.0] and [require_alias] request flag is [true] and [.kibana_8.11.0] is not an alias
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:479:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
at ClientTraced.UpdateApi [as update] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/update.js:50:12)
[2023-11-27T18:04:00.922+00:00][WARN ][plugins.usageCollection.usage-collection.usage-counters-service] ResponseError: Saved object index alias [.kibana_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_8.11.0] and [require_alias] request flag is [true] and [.kibana_8.11.0] is not an alias
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:479:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
at ClientTraced.UpdateApi [as update] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/update.js:50:12)
[2023-11-27T18:04:00.924+00:00][ERROR][plugins.alerting] Failed to simulate index template mappings for .alerts-observability.uptime.alerts-default-index-template; not applying mappings - invalid_index_template_exception
Root causes:
invalid_index_template_exception: index_template [.alerts-observability.uptime.alerts-default-index-template] invalid, cause [index template [.alerts-observability.uptime.alerts-default-index-template] specifies component templates [.alerts-legacy-alert-mappings] that do not exist]
[2023-11-27T18:04:00.925+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-observability.uptime.alerts-default-000001
[2023-11-27T18:04:00.932+00:00][WARN ][plugins.usageCollection.usage-collection.usage-counters-service] ResponseError: Saved object index alias [.kibana_8.11.0] not found: index_not_found_exception
Root causes:
index_not_found_exception: no such index [.kibana_8.11.0] and [require_alias] request flag is [true] and [.kibana_8.11.0] is not an alias
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:479:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
at ClientTraced.UpdateApi [as update] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/update.js:50:12)
[2023-11-27T18:04:01.046+00:00][INFO ][plugins.eventLog] Creating datastream .kibana-event-log-ds
[2023-11-27T18:04:01.490+00:00][INFO ][plugins.fleet.fleet:check-deleted-files-task:1.0.1] [runTask()] started
[2023-11-27T18:04:01.789+00:00][INFO ][plugins.alerting] Creating concrete write index - .internal.alerts-ml.anomaly-detection.alerts-default-000001
[2023-11-27T18:04:01.803+00:00][INFO ][plugins.fleet.fleet:check-deleted-files-task:1.0.1] [runTask()] ended: no files to process
[2023-11-27T18:04:02.156+00:00][INFO ][plugins.observabilityAIAssistant.service] Creating concrete write index - .kibana-observability-ai-assistant-kb-000001
[2023-11-27T18:04:02.340+00:00][INFO ][plugins.observabilityAIAssistant.service] Successfully set up index assets
[2023-11-27T18:04:02.607+00:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"inactive":0,"unenrolled":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-11-27T18:04:02.794+00:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/node_modules/@kbn/screenshotting-plugin/chromium/headless_shell-linux_x64/headless_shell
[2023-11-27T18:04:04.420+00:00][INFO ][status] Kibana is now available (was degraded)
here is the elasticsearch deployment
```
apiVersion: apps/v1
kind: Deployment
metadata:
name: elasticsearch
spec:
replicas: 2
selector:
matchLabels:
app: elasticsearch
template:
metadata:
labels:
app: elasticsearch
spec:
containers:
- name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
resources:
limits:
memory: "2Gi"
requests:
memory: "1Gi"
ports:
- containerPort: 9200
- containerPort: 9300
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
- name: config
mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
subPath: elasticsearch.yml
volumes:
- name: data
emptyDir: {}
- name: config
configMap:
name: elasticsearch-config