First:
The problem is solved, the SSL connection works now.
I think I have to summarize all this in a separate thread in the next few days.. 
I´ve received these messages:
[2023-08-18T13:34:48.329+02:00][DEBUG][elasticsearch.query.data] 401 - 615.0B
GET /_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip [security_exception]: missing authentication credentials for REST request [/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip]
[2023-08-18T13:34:48.938+02:00][DEBUG][elasticsearch.query.data] 401 - 459.0B
GET / [security_exception]: missing authentication credentials for REST request [/]
I read in an article that you have to enter the username and password in the kibana.yml:
After trying this in plain text with the Elastic user, I received an error message:
FATAL Error: [config validation of [elasticsearch].username]: value of "elastic" is forbidden. This is a superuser account that cannot write to system indices that Kibana needs to function. Use a service account token instead. Learn more: https://www.elastic.co/guide/en/elasticsearch/reference/8.0/service-accounts.html
With this message and this web address I created the token:
C:\_Mon\e\bin>elasticsearch-service-tokens create elastic/kibana service-token-01
SERVICE_TOKEN elastic/kibana/service-token-01 = AAEAA...lBX09oZw
In the next step I then tried to test this with Curl
C:\Users\273872a\Documents\curl\bin>curl -H "Authorization: Bearer AAEAA...lBX09oZw" https://wxtask2p.idm.lan.local:9200/_cluster/health
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
This failed because I didn´t specify the certificate.
Second try:
C:\Users\273872a\Documents\curl\bin>curl --cacert c:\_Mon\e\config\certs\wxtask2p-idm-lan-local.crt -H "Authorization: Bearer AAEAA...lBX09oZww" https://wxtask2p.idm.lan.local:9200/_cluster/health
{"cluster_name":"elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":60,"active_shards":60,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":12,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":83.33333333333334}
After that I restarted elasticsearch (although it was probably not necessary) and then kibana.
And suddenly everything works. The connection is secured and logging in also works without problems.
However, the kibana system shows me some strange and long messages.
QUESTION:
Is this normal?
[2023-08-18T14:21:20.099+02:00][DEBUG][elasticsearch.query.data] 200 - 227.0B
POST /.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true
{"query":{"bool":{"must":[{"term":{"type":"task"}},{"bool":{"must":[{"bool":{"must":[{"term":{"task.enabled":true}}]}},{"bool":{"should":[{"bool":{"must":[{"term":{"task.status":"idle"}},{"range":{"task.runAt":{"lte":"now"}}}]}},{"bool":{"must":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}]}},{"range":{"task.retryAt":{"lte":"now"}}}]}}]}}],"filter":[{"bool":{"must_not":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}],"must":{"range":{"task.retryAt":{"gt":"now"}}}}}]}}]}}]}},"script":{"source":"\n if (params.claimableTaskTypes.contains(ctx._source.task.taskType)) {\n if (ctx._source.task.schedule != null || ctx._source.task.attempts < params.taskMaxAttempts[ctx._source.task.taskType]) {\n if(ctx._source.task.retryAt != null && ZonedDateTime.parse(ctx._source.task.retryAt).toInstant().toEpochMilli() < params.now) {\n ctx._source.task.scheduledAt=ctx._source.task.retryAt;\n } else {\n ctx._source.task.scheduledAt=ctx._source.task.runAt;\n }\n ctx._source.task.status = \"claiming\"; ctx._source.task.ownerId=params.fieldUpdates.ownerId; ctx._source.task.retryAt=params.fieldUpdates.retryAt;\n } else {\n ctx._source.task.status = \"failed\";\n }\n } else if (params.unusedTaskTypes.contains(ctx._source.task.taskType)) {\n ctx._source.task.status = \"unrecognized\";\n } else {\n ctx.op = \"noop\";\n }","lang":"painless","params":{"now":1692361280094,"fieldUpdates":{"ownerId":"kibana:3cc50870-5b0a-4625-a563-055064a00ef7","retryAt":"2023-08-18T12:21:50.073Z"},"claimableTaskTypes":["session_cleanup","actions_telemetry","cleanup_failed_action_executions","alerting_telemetry","alerts_invalidate_api_keys","alerting_health_check","reports:monitor","alerting:transform_health","actions:.email","actions:.index","actions:.pagerduty","actions:.swimlane","actions:.server-log","actions:.slack","actions:.webhook","actions:.cases-webhook","actions:.xmatters","actions:.servicenow","actions:.servicenow-sir","actions:.servicenow-itom","actions:.jira","actions:.resilient","actions:.teams","actions:.torq","actions:.opsgenie","actions:.tines","alerting:.index-threshold","alerting:.geo-containment","alerting:.es-query","dashboard_telemetry","cases-telemetry-task","Fleet-Usage-Sender","Fleet-Usage-Logger","fleet:reassign_action:retry","fleet:unenroll_action:retry","fleet:upgrade_action:retry","fleet:update_agent_tags:retry","fleet:request_diagnostics:retry","fleet:check-deleted-files-task","osquery:telemetry-packs","osquery:telemetry-saved-queries","osquery:telemetry-configs","cloud_security_posture-stats_task","ML:saved-objects-sync","alerting:xpack.ml.anomaly_detection_alert","alerting:xpack.ml.anomaly_detection_jobs_health","UPTIME:SyntheticsService:Sync-Saved-Monitor-Objects","alerting:xpack.uptime.alerts.monitorStatus","alerting:xpack.uptime.alerts.tlsCertificate","alerting:xpack.uptime.alerts.durationAnomaly","alerting:xpack.uptime.alerts.tls","alerting:xpack.synthetics.alerts.monitorStatus","alerting:siem.eqlRule","alerting:siem.savedQueryRule","alerting:siem.indicatorRule","alerting:siem.mlRule","alerting:siem.queryRule","alerting:siem.thresholdRule","alerting:siem.newTermsRule","alerting:siem.notifications","endpoint:user-artifact-packager","security:endpoint-diagnostics","security:endpoint-meta-telemetry","security:telemetry-lists","security:telemetry-detection-rules","security:telemetry-prebuilt-rule-alerts","security:telemetry-timelines","security:telemetry-configuration","security:telemetry-filterlist-artifact","endpoint:metadata-check-transforms-task","alerting:metrics.alert.anomaly","alerting:logs.alert.document.count","alerting:metrics.alert.inventory.threshold","alerting:metrics.alert.threshold","alerting:monitoring_alert_cluster_health","alerting:monitoring_alert_license_expiration","alerting:monitoring_alert_cpu_usage","alerting:monitoring_alert_missing_monitoring_data","alerting:monitoring_alert_disk_usage","alerting:monitoring_alert_thread_pool_search_rejections","alerting:monitoring_alert_thread_pool_write_rejections","alerting:monitoring_alert_jvm_memory_usage","alerting:monitoring_alert_nodes_changed","alerting:monitoring_alert_logstash_version_mismatch","alerting:monitoring_alert_kibana_version_mismatch","alerting:monitoring_alert_elasticsearch_version_mismatch","alerting:monitoring_ccr_read_exceptions","alerting:monitoring_shard_size","apm-telemetry-task","alerting:apm.transaction_duration","alerting:apm.anomaly","alerting:apm.error_rate","alerting:apm.transaction_error_rate"],"skippedTaskTypes":["report:execute","apm-source-map-migration-task"],"unusedTaskTypes":["sampleTaskRemovedType","alerting:siem.signals","search_sessions_monitor","search_sessions_cleanup","search_sessions_expire"],"taskMaxAttempts":{"session_cleanup":3,"actions_telemetry":3,"cleanup_failed_action_executions":3,"alerting_telemetry":3,"alerts_invalidate_api_keys":3,"alerting_health_check":3,"reports:monitor":1,"alerting:transform_health":3,"actions:.email":3,"actions:.index":3,"actions:.pagerduty":3,"actions:.swimlane":3,"actions:.server-log":3,"actions:.slack":3,"actions:.webhook":3,"actions:.cases-webhook":3,"actions:.xmatters":3,"actions:.servicenow":3,"actions:.servicenow-sir":3,"actions:.servicenow-itom":3,"actions:.jira":3,"actions:.resilient":3,"actions:.teams":3,"actions:.torq":3,"actions:.opsgenie":3,"actions:.tines":3,"alerting:.index-threshold":3,"alerting:.geo-containment":3,"alerting:.es-query":3,"dashboard_telemetry":3,"cases-telemetry-task":3,"Fleet-Usage-Sender":1,"Fleet-Usage-Logger":1,"fleet:reassign_action:retry":1,"fleet:unenroll_action:retry":1,"fleet:upgrade_action:retry":1,"fleet:update_agent_tags:retry":1,"fleet:request_diagnostics:retry":1,"fleet:check-deleted-files-task":3,"osquery:telemetry-packs":3,"osquery:telemetry-saved-queries":3,"osquery:telemetry-configs":3,"cloud_security_posture-stats_task":3,"ML:saved-objects-sync":3,"alerting:xpack.ml.anomaly_detection_alert":3,"alerting:xpack.ml.anomaly_detection_jobs_health":3,"UPTIME:SyntheticsService:Sync-Saved-Monitor-Objects":3,"alerting:xpack.uptime.alerts.monitorStatus":3,"alerting:xpack.uptime.alerts.tlsCertificate":3,"alerting:xpack.uptime.alerts.durationAnomaly":3,"alerting:xpack.uptime.alerts.tls":3,"alerting:xpack.synthetics.alerts.monitorStatus":3,"alerting:siem.eqlRule":3,"alerting:siem.savedQueryRule":3,"alerting:siem.indicatorRule":3,"alerting:siem.mlRule":3,"alerting:siem.queryRule":3,"alerting:siem.thresholdRule":3,"alerting:siem.newTermsRule":3,"alerting:siem.notifications":3,"endpoint:user-artifact-packager":3,"security:endpoint-diagnostics":3,"security:endpoint-meta-telemetry":3,"security:telemetry-lists":3,"security:telemetry-detection-rules":3,"security:telemetry-prebuilt-rule-alerts":3,"security:telemetry-timelines":3,"security:telemetry-configuration":3,"security:telemetry-filterlist-artifact":3,"endpoint:metadata-check-transforms-task":3,"alerting:metrics.alert.anomaly":3,"alerting:logs.alert.document.count":3,"alerting:metrics.alert.inventory.threshold":3,"alerting:metrics.alert.threshold":3,"alerting:monitoring_alert_cluster_health":3,"alerting:monitoring_alert_license_expiration":3,"alerting:monitoring_alert_cpu_usage":3,"alerting:monitoring_alert_missing_monitoring_data":3,"alerting:monitoring_alert_disk_usage":3,"alerting:monitoring_alert_thread_pool_search_rejections":3,"alerting:monitoring_alert_thread_pool_write_rejections":3,"alerting:monitoring_alert_jvm_memory_usage":3,"alerting:monitoring_alert_nodes_changed":3,"alerting:monitoring_alert_logstash_version_mismatch":3,"alerting:monitoring_alert_kibana_version_mismatch":3,"alerting:monitoring_alert_elasticsearch_version_mismatch":3,"alerting:monitoring_ccr_read_exceptions":3,"alerting:monitoring_shard_size":3,"apm-telemetry-task":3,"alerting:apm.transaction_duration":3,"alerting:apm.anomaly":3,"alerting:apm.error_rate":3,"alerting:apm.transaction_error_rate":3}}},"sort":[{"_script":{"type":"number","order":"asc","script":{"lang":"painless","source":"\nif (doc['task.retryAt'].size()!=0) {\n return doc['task.retryAt'].value.toInstant().toEpochMilli();\n}\nif (doc['task.runAt'].size()!=0) {\n return doc['task.runAt'].value.toInstant().toEpochMilli();\n}\n "}}}],"max_docs":10,"conflicts":"proceed"}
[2023-08-18T14:21:20.591+02:00][DEBUG][elasticsearch.query.data] 200 - 125.0B
GET /_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip
[2023-08-18T14:21:20.667+02:00][ERROR][plugins.fleet] Failed to fetch latest version of synthetics from registry: Error connecting to package registry: request to https://epr.elastic.co/search?package=synthetics&prerelease=true&kibana.version=8.7.0 failed, reason: connect ETIMEDOUT 34.120.127.130:443
[2023-08-18T14:21:20.671+02:00][DEBUG][elasticsearch.query.data] 200 - 10.3KB
GET /.kibana_8.7.0/_doc/epm-packages%3Asynthetics
[2023-08-18T14:21:20.671+02:00][INFO ][plugins.synthetics] Installed synthetics index templates
[2023-08-18T14:21:22.905+02:00][DEBUG][elasticsearch.query.monitoring] 200 - 1.3KB
GET /_xpack
[2023-08-18T14:21:22.937+02:00][DEBUG][elasticsearch.query.data] 200 - 399.0B
GET /.kibana_8.7.0/_doc/telemetry%3Atelemetry
[2023-08-18T14:21:23.016+02:00][DEBUG][metrics.ops] memory: 275.0MB uptime: 0:02:02 load: [0.00,0.00,0.00] mean delay: 15.725 delay histogram: { 50: 15.630; 95: 16.163; 99: 24.134 }
[2023-08-18T14:21:23.094+02:00][DEBUG][elasticsearch.query.data] 200 - 125.0B
GET /_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip
[2023-08-18T14:21:23.099+02:00][DEBUG][elasticsearch.query.data] 200 - 227.0B
POST /.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true
{"query":{"bool":{"must":[{"term":{"type":"task"}},{"bool":{"must":[{"bool":{"must":[{"term":{"task.enabled":true}}]}},{"bool":{"should":[{"bool":{"must":[{"term":{"task.status":"idle"}},{"range":{"task.runAt":{"lte":"now"}}}]}},{"bool":{"must":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}]}},{"range":{"task.retryAt":{"lte":"now"}}}]}}]}}],"filter":[{"bool":{"must_not":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}],"must":{"range":{"task.retryAt":{"gt":"now"}}}}}]}}]}}]}},"script":{"source":"\n if (params.claimableTaskTypes.contains(ctx._source.task.taskType)) {\n if (ctx._source.task.schedule != null || ctx._source.task.attempts < params.taskMaxAttempts[ctx._source.task.taskType]) {\n if(ctx._source.task.retryAt != null && ZonedDateTime.parse(ctx._source.task.retryAt).toInstant().toEpochMilli() < params.now) {\n ctx._source.task.scheduledAt=ctx._source.task.retryAt;\n } else {\n ctx._source.task.scheduledAt=ctx._source.task.runAt;\n }\n ctx._source.task.status = \"claiming\"; ctx._source.task.ownerId=params.fieldUpdates.ownerId; ctx._source.task.retryAt=params.fieldUpdates.retryAt;\n } else {\n ctx._source.task.status = \"failed\";\n }\n } else if (params.unusedTaskTypes.contains(ctx._source.task.taskType)) {\n ctx._source.task.status = \"unrecognized\";\n } else {\n ctx.op = \"noop\";\n }","lang":"painless","params":{"now":1692361283091,"fieldUpdates":{"ownerId":"kibana:3cc50870-5b0a-4625-a563-055064a00ef7","retryAt":"2023-08-18T12:21:53.090Z"},"claimableTaskTypes":["apm-source-map-migration-task"],"skippedTaskTypes":["session_cleanup","actions_telemetry","cleanup_failed_action_executions","alerting_telemetry","alerts_invalidate_api_keys","alerting_health_check","report:execute","reports:monitor","alerting:transform_health","actions:.email","actions:.index","actions:.pagerduty","actions:.swimlane","actions:.server-log","actions:.slack","actions:.webhook","actions:.cases-webhook","actions:.xmatters","actions:.servicenow","actions:.servicenow-sir","actions:.servicenow-itom","actions:.jira","actions:.resilient","actions:.teams","actions:.torq","actions:.opsgenie","actions:.tines","alerting:.index-threshold","alerting:.geo-containment","alerting:.es-query","dashboard_telemetry","cases-telemetry-task","Fleet-Usage-Sender","Fleet-Usage-Logger","fleet:reassign_action:retry","fleet:unenroll_action:retry","fleet:upgrade_action:retry","fleet:update_agent_tags:retry","fleet:request_diagnostics:retry","fleet:check-deleted-files-task","osquery:telemetry-packs","osquery:telemetry-saved-queries","osquery:telemetry-configs","cloud_security_posture-stats_task","ML:saved-objects-sync","alerting:xpack.ml.anomaly_detection_alert","alerting:xpack.ml.anomaly_detection_jobs_health","UPTIME:SyntheticsService:Sync-Saved-Monitor-Objects","alerting:xpack.uptime.alerts.monitorStatus","alerting:xpack.uptime.alerts.tlsCertificate","alerting:xpack.uptime.alerts.durationAnomaly","alerting:xpack.uptime.alerts.tls","alerting:xpack.synthetics.alerts.monitorStatus","alerting:siem.eqlRule","alerting:siem.savedQueryRule","alerting:siem.indicatorRule","alerting:siem.mlRule","alerting:siem.queryRule","alerting:siem.thresholdRule","alerting:siem.newTermsRule","alerting:siem.notifications","endpoint:user-artifact-packager","security:endpoint-diagnostics","security:endpoint-meta-telemetry","security:telemetry-lists","security:telemetry-detection-rules","security:telemetry-prebuilt-rule-alerts","security:telemetry-timelines","security:telemetry-configuration","security:telemetry-filterlist-artifact","endpoint:metadata-check-transforms-task","alerting:metrics.alert.anomaly","alerting:logs.alert.document.count","alerting:metrics.alert.inventory.threshold","alerting:metrics.alert.threshold","alerting:monitoring_alert_cluster_health","alerting:monitoring_alert_license_expiration","alerting:monitoring_alert_cpu_usage","alerting:monitoring_alert_missing_monitoring_data","alerting:monitoring_alert_disk_usage","alerting:monitoring_alert_thread_pool_search_rejections","alerting:monitoring_alert_thread_pool_write_rejections","alerting:monitoring_alert_jvm_memory_usage","alerting:monitoring_alert_nodes_changed","alerting:monitoring_alert_logstash_version_mismatch","alerting:monitoring_alert_kibana_version_mismatch","alerting:monitoring_alert_elasticsearch_version_mismatch","alerting:monitoring_ccr_read_exceptions","alerting:monitoring_shard_size","apm-telemetry-task","alerting:apm.transaction_duration","alerting:apm.anomaly","alerting:apm.error_rate","alerting:apm.transaction_error_rate"],"unusedTaskTypes":["sampleTaskRemovedType","alerting:siem.signals","search_sessions_monitor","search_sessions_cleanup","search_sessions_expire"],"taskMaxAttempts":{"apm-source-map-migration-task":5}}},"sort":[{"_script":{"type":"number","order":"asc","script":{"lang":"painless","source":"\nif (doc['task.retryAt'].size()!=0) {\n return doc['task.retryAt'].value.toInstant().toEpochMilli();\n}\nif (doc['task.runAt'].size()!=0) {\n return doc['task.runAt'].value.toInstant().toEpochMilli();\n}\n "}}}],"max_docs":1,"conflicts":"proceed"}
[2023-08-18T14:21:23.169+02:00][DEBUG][plugins.monitoring.monitoring.kibana-monitoring] not sending [kibana_settings] monitoring document because [undefined] is null or invalid.
[2023-08-18T14:21:23.170+02:00][DEBUG][plugins.monitoring.monitoring.kibana-monitoring] Uploading bulk stats payload to the local cluster
[2023-08-18T14:21:23.175+02:00][DEBUG][elasticsearch.query.data] 200 - 228.0B
POST /.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true
{"query":{"bool":{"must":[{"term":{"type":"task"}},{"bool":{"must":[{"bool":{"must":[{"term":{"task.enabled":true}}]}},{"bool":{"should":[{"bool":{"must":[{"term":{"task.status":"idle"}},{"range":{"task.runAt":{"lte":"now"}}}]}},{"bool":{"must":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}]}},{"range":{"task.retryAt":{"lte":"now"}}}]}}]}}],"filter":[{"bool":{"must_not":[{"bool":{"should":[{"term":{"task.status":"running"}},{"term":{"task.status":"claiming"}}],"must":{"range":{"task.retryAt":{"gt":"now"}}}}}]}}]}}]}},"script":{"source":"\n if (params.claimableTaskTypes.contains(ctx._source.task.taskType)) {\n if (ctx._source.task.schedule != null || ctx._source.task.attempts < params.taskMaxAttempts[ctx._source.task.taskType]) {\n if(ctx._source.task.retryAt != null && ZonedDateTime.parse(ctx._source.task.retryAt).toInstant().toEpochMilli() < params.now) {\n ctx._source.task.scheduledAt=ctx._source.task.retryAt;\n } else {\n ctx._source.task.scheduledAt=ctx._source.task.runAt;\n }\n ctx._source.task.status = \"claiming\"; ctx._source.task.ownerId=params.fieldUpdates.ownerId; ctx._source.task.retryAt=params.fieldUpdates.retryAt;\n } else {\n ctx._source.task.status = \"failed\";\n }\n } else if (params.unusedTaskTypes.contains(ctx._source.task.taskType)) {\n ctx._source.task.status = \"unrecognized\";\n } else {\n ctx.op = \"noop\";\n }","lang":"painless","params":{"now":1692361283100,"fieldUpdates":{"ownerId":"kibana:3cc50870-5b0a-4625-a563-055064a00ef7","retryAt":"2023-08-18T12:21:53.090Z"},"claimableTaskTypes":["session_cleanup","actions_telemetry","cleanup_failed_action_executions","alerting_telemetry","alerts_invalidate_api_keys","alerting_health_check","reports:monitor","alerting:transform_health","actions:.email","actions:.index","actions:.pagerduty","actions:.swimlane","actions:.server-log","actions:.slack","actions:.webhook","actions:.cases-webhook","actions:.xmatters","actions:.servicenow","actions:.servicenow-sir","actions:.servicenow-itom","actions:.jira","actions:.resilient","actions:.teams","actions:.torq","actions:.opsgenie","actions:.tines","alerting:.index-threshold","alerting:.geo-containment","alerting:.es-query","dashboard_telemetry","cases-telemetry-task","Fleet-Usage-Sender","Fleet-Usage-Logger","fleet:reassign_action:retry","fleet:unenroll_action:retry","fleet:upgrade_action:retry","fleet:update_agent_tags:retry","fleet:request_diagnostics:retry","fleet:check-deleted-files-task","osquery:telemetry-packs","osquery:telemetry-saved-queries","osquery:telemetry-configs","cloud_security_posture-stats_task","ML:saved-objects-sync","alerting:xpack.ml.anomaly_detection_alert","alerting:xpack.ml.anomaly_detection_jobs_health","UPTIME:SyntheticsService:Sync-Saved-Monitor-Objects","alerting:xpack.uptime.alerts.monitorStatus","alerting:xpack.uptime.alerts.tlsCertificate","alerting:xpack.uptime.alerts.durationAnomaly","alerting:xpack.uptime.alerts.tls","alerting:xpack.synthetics.alerts.monitorStatus","alerting:siem.eqlRule","alerting:siem.savedQueryRule","alerting:siem.indicatorRule","alerting:siem.mlRule","alerting:siem.queryRule","alerting:siem.thresholdRule","alerting:siem.newTermsRule","alerting:siem.notifications","endpoint:user-artifact-packager","security:endpoint-diagnostics","security:endpoint-meta-telemetry","security:telemetry-lists","security:telemetry-detection-rules","security:telemetry-prebuilt-rule-alerts","security:telemetry-timelines","security:telemetry-configuration","security:telemetry-filterlist-artifact","endpoint:metadata-check-transforms-task","alerting:metrics.alert.anomaly","alerting:logs.alert.document.count","alerting:metrics.alert.inventory.threshold","alerting:metrics.alert.threshold","alerting:monitoring_alert_cluster_health","alerting:monitoring_alert_license_expiration","alerting:monitoring_alert_cpu_usage","alerting:monitoring_alert_missing_monitoring_data","alerting:monitoring_alert_disk_usage","alerting:monitoring_alert_thread_pool_search_rejections","alerting:monitoring_alert_thread_pool_write_rejections","alerting:monitoring_alert_jvm_memory_usage","alerting:monitoring_alert_nodes_changed","alerting:monitoring_alert_logstash_version_mismatch","alerting:monitoring_alert_kibana_version_mismatch","alerting:monitoring_alert_elasticsearch_version_mismatch","alerting:monitoring_ccr_read_exceptions","alerting:monitoring_shard_size","apm-telemetry-task","alerting:apm.transaction_duration","alerting:apm.anomaly","alerting:apm.error_rate","alerting:apm.transaction_error_rate"],"skippedTaskTypes":["report:execute","apm-source-map-migration-task"],"unusedTaskTypes":["sampleTaskRemovedType","alerting:siem.signals","search_sessions_monitor","search_sessions_cleanup","search_sessions_expire"],"taskMaxAttempts":{"session_cleanup":3,"actions_telemetry":3,"cleanup_failed_action_executions":3,"alerting_telemetry":3,"alerts_invalidate_api_keys":3,"alerting_health_check":3,"reports:monitor":1,"alerting:transform_health":3,"actions:.email":3,"actions:.index":3,"actions:.pagerduty":3,"actions:.swimlane":3,"actions:.server-log":3,"actions:.slack":3,"actions:.webhook":3,"actions:.cases-webhook":3,"actions:.xmatters":3,"actions:.servicenow":3,"actions:.servicenow-sir":3,"actions:.servicenow-itom":3,"actions:.jira":3,"actions:.resilient":3,"actions:.teams":3,"actions:.torq":3,"actions:.opsgenie":3,"actions:.tines":3,"alerting:.index-threshold":3,"alerting:.geo-containment":3,"alerting:.es-query":3,"dashboard_telemetry":3,"cases-telemetry-task":3,"Fleet-Usage-Sender":1,"Fleet-Usage-Logger":1,"fleet:reassign_action:retry":1,"fleet:unenroll_action:retry":1,"fleet:upgrade_action:retry":1,"fleet:update_agent_tags:retry":1,"fleet:request_diagnostics:retry":1,"fleet:check-deleted-files-task":3,"osquery:telemetry-packs":3,"osquery:telemetry-saved-queries":3,"osquery:telemetry-configs":3,"cloud_security_posture-stats_task":3,"ML:saved-objects-sync":3,"alerting:xpack.ml.anomaly_detection_alert":3,"alerting:xpack.ml.anomaly_detection_jobs_health":3,"UPTIME:SyntheticsService:Sync-Saved-Monitor-Objects":3,"alerting:xpack.uptime.alerts.monitorStatus":3,"alerting:xpack.uptime.alerts.tlsCertificate":3,"alerting:xpack.uptime.alerts.durationAnomaly":3,"alerting:xpack.uptime.alerts.tls":3,"alerting:xpack.synthetics.alerts.monitorStatus":3,"alerting:siem.eqlRule":3,"alerting:siem.savedQueryRule":3,"alerting:siem.indicatorRule":3,"alerting:siem.mlRule":3,"alerting:siem.queryRule":3,"alerting:siem.thresholdRule":3,"alerting:siem.newTermsRule":3,"alerting:siem.notifications":3,"endpoint:user-artifact-packager":3,"security:endpoint-diagnostics":3,"security:endpoint-meta-telemetry":3,"security:telemetry-lists":3,"security:telemetry-detection-rules":3,"security:telemetry-prebuilt-rule-alerts":3,"security:telemetry-timelines":3,"security:telemetry-configuration":3,"security:telemetry-filterlist-artifact":3,"endpoint:metadata-check-transforms-task":3,"alerting:metrics.alert.anomaly":3,"alerting:logs.alert.document.count":3,"alerting:metrics.alert.inventory.threshold":3,"alerting:metrics.alert.threshold":3,"alerting:monitoring_alert_cluster_health":3,"alerting:monitoring_alert_license_expiration":3,"alerting:monitoring_alert_cpu_usage":3,"alerting:monitoring_alert_missing_monitoring_data":3,"alerting:monitoring_alert_disk_usage":3,"alerting:monitoring_alert_thread_pool_search_rejections":3,"alerting:monitoring_alert_thread_pool_write_rejections":3,"alerting:monitoring_alert_jvm_memory_usage":3,"alerting:monitoring_alert_nodes_changed":3,"alerting:monitoring_alert_logstash_version_mismatch":3,"alerting:monitoring_alert_kibana_version_mismatch":3,"alerting:monitoring_alert_elasticsearch_version_mismatch":3,"alerting:monitoring_ccr_read_exceptions":3,"alerting:monitoring_shard_size":3,"apm-telemetry-task":3,"alerting:apm.transaction_duration":3,"alerting:apm.anomaly":3,"alerting:apm.error_rate":3,"alerting:apm.transaction_error_rate":3}}},"sort":[{"_script":{"type":"number","order":"asc","script":{"lang":"painless","source":"\nif (doc['task.retryAt'].size()!=0) {\n return doc['task.retryAt'].value.toInstant().toEpochMilli();\n}\nif (doc['task.runAt'].size()!=0) {\n return doc['task.runAt'].value.toInstant().toEpochMilli();\n}\n "}}}],"max_docs":10,"conflicts":"proceed"}
[2023-08-18T14:21:23.182+02:00][DEBUG][elasticsearch.query.data] 200 - 898.0B
POST /.kibana_task_manager/_search?ignore_unavailable=true
{"query":{"bool":{"must":[{"term":{"type":"task"}},{"bool":{"must":[{"term":{"task.ownerId":"kibana:3cc50870-5b0a-4625-a563-055064a00ef7"}},{"term":{"task.status":"claiming"}},{"bool":{"should":[{"term":{"task.taskType":"session_cleanup"}},{"term":{"task.taskType":"actions_telemetry"}},{"term":{"task.taskType":"cleanup_failed_action_executions"}},{"term":{"task.taskType":"alerting_telemetry"}},{"term":{"task.taskType":"alerts_invalidate_api_keys"}},{"term":{"task.taskType":"alerting_health_check"}},{"term":{"task.taskType":"reports:monitor"}},{"term":{"task.taskType":"alerting:transform_health"}},{"term":{"task.taskType":"actions:.email"}},{"term":{"task.taskType":"actions:.index"}},{"term":{"task.taskType":"actions:.pagerduty"}},{"term":{"task.taskType":"actions:.swimlane"}},{"term":{"task.taskType":"actions:.server-log"}},{"term":{"task.taskType":"actions:.slack"}},{"term":{"task.taskType":"actions:.webhook"}},{"term":{"task.taskType":"actions:.cases-webhook"}},{"term":{"task.taskType":"actions:.xmatters"}},{"term":{"task.taskType":"actions:.servicenow"}},{"term":{"task.taskType":"actions:.servicenow-sir"}},{"term":{"task.taskType":"actions:.servicenow-itom"}},{"term":{"task.taskType":"actions:.jira"}},{"term":{"task.taskType":"actions:.resilient"}},{"term":{"task.taskType":"actions:.teams"}},{"term":{"task.taskType":"actions:.torq"}},{"term":{"task.taskType":"actions:.opsgenie"}},{"term":{"task.taskType":"actions:.tines"}},{"term":{"task.taskType":"alerting:.index-threshold"}},{"term":{"task.taskType":"alerting:.geo-containment"}},{"term":{"task.taskType":"alerting:.es-query"}},{"term":{"task.taskType":"dashboard_telemetry"}},{"term":{"task.taskType":"cases-telemetry-task"}},{"term":{"task.taskType":"Fleet-Usage-Sender"}},{"term":{"task.taskType":"Fleet-Usage-Logger"}},{"term":{"task.taskType":"fleet:reassign_action:retry"}},{"term":{"task.taskType":"fleet:unenroll_action:retry"}},{"term":{"task.taskType":"fleet:upgrade_action:retry"}},{"term":{"task.taskType":"fleet:update_agent_tags:retry"}},{"term":{"task.taskType":"fleet:request_diagnostics:retry"}},{"term":{"task.taskType":"fleet:check-deleted-files-task"}},{"term":{"task.taskType":"osquery:telemetry-packs"}},{"term":{"task.taskType":"osquery:telemetry-saved-queries"}},{"term":{"task.taskType":"osquery:telemetry-configs"}},{"term":{"task.taskType":"cloud_security_posture-stats_task"}},{"term":{"task.taskType":"ML:saved-objects-sync"}},{"term":{"task.taskType":"alerting:xpack.ml.anomaly_detection_alert"}},{"term":{"task.taskType":"alerting:xpack.ml.anomaly_detection_jobs_health"}},{"term":{"task.taskType":"UPTIME:SyntheticsService:Sync-Saved-Monitor-Objects"}},{"term":{"task.taskType":"alerting:xpack.uptime.alerts.monitorStatus"}},{"term":{"task.taskType":"alerting:xpack.uptime.alerts.tlsCertificate"}},{"term":{"task.taskType":"alerting:xpack.uptime.alerts.durationAnomaly"}},{"term":{"task.taskType":"alerting:xpack.uptime.alerts.tls"}},{"term":{"task.taskType":"alerting:xpack.synthetics.alerts.monitorStatus"}},{"term":{"task.taskType":"alerting:siem.eqlRule"}},{"term":{"task.taskType":"alerting:siem.savedQueryRule"}},{"term":{"task.taskType":"alerting:siem.indicatorRule"}},{"term":{"task.taskType":"alerting:siem.mlRule"}},{"term":{"task.taskType":"alerting:siem.queryRule"}},{"term":{"task.taskType":"alerting:siem.thresholdRule"}},{"term":{"task.taskType":"alerting:siem.newTermsRule"}},{"term":{"task.taskType":"alerting:siem.notifications"}},{"term":{"task.taskType":"endpoint:user-artifact-packager"}},{"term":{"task.taskType":"security:endpoint-diagnostics"}},{"term":{"task.taskType":"security:endpoint-meta-telemetry"}},{"term":{"task.taskType":"security:telemetry-lists"}},{"term":{"task.taskType":"security:telemetry-detection-rules"}},{"term":{"task.taskType":"security:telemetry-prebuilt-rule-alerts"}},{"term":{"task.taskType":"security:telemetry-timelines"}},{"term":{"task.taskType":"security:telemetry-configuration"}},{"term":{"task.taskType":"security:telemetry-filterlist-artifact"}},{"term":{"task.taskType":"endpoint:metadata-check-transforms-task"}},{"term":{"task.taskType":"alerting:metrics.alert.anomaly"}},{"term":{"task.taskType":"alerting:logs.alert.document.count"}},{"term":{"task.taskType":"alerting:metrics.alert.inventory.threshold"}},{"term":{"task.taskType":"alerting:metrics.alert.threshold"}},{"term":{"task.taskType":"alerting:monitoring_alert_cluster_health"}},{"term":{"task.taskType":"alerting:monitoring_alert_license_expiration"}},{"term":{"task.taskType":"alerting:monitoring_alert_cpu_usage"}},{"term":{"task.taskType":"alerting:monitoring_alert_missing_monitoring_data"}},{"term":{"task.taskType":"alerting:monitoring_alert_disk_usage"}},{"term":{"task.taskType":"alerting:monitoring_alert_thread_pool_search_rejections"}},{"term":{"task.taskType":"alerting:monitoring_alert_thread_pool_write_rejections"}},{"term":{"task.taskType":"alerting:monitoring_alert_jvm_memory_usage"}},{"term":{"task.taskType":"alerting:monitoring_alert_nodes_changed"}},{"term":{"task.taskType":"alerting:monitoring_alert_logstash_version_mismatch"}},{"term":{"task.taskType":"alerting:monitoring_alert_kibana_version_mismatch"}},{"term":{"task.taskType":"alerting:monitoring_alert_elasticsearch_version_mismatch"}},{"term":{"task.taskType":"alerting:monitoring_ccr_read_exceptions"}},{"term":{"task.taskType":"alerting:monitoring_shard_size"}},{"term":{"task.taskType":"apm-telemetry-task"}},{"term":{"task.taskType":"alerting:apm.transaction_duration"}},{"term":{"task.taskType":"alerting:apm.anomaly"}},{"term":{"task.taskType":"alerting:apm.error_rate"}},{"term":{"task.taskType":"alerting:apm.transaction_error_rate"}}]}}]}}]}},"size":10,"seq_no_primary_term":true,"sort":{"_script":{"type":"number","order":"asc","script":{"lang":"painless","source":"\nif (doc['task.retryAt'].size()!=0) {\n return doc['task.retryAt'].value.toInstant().toEpochMilli();\n}\nif (doc['task.runAt'].size()!=0) {\n return doc['task.runAt'].value.toInstant().toEpochMilli();\n}\n "}}}}