Kibana bouncing degraded - available

wrsnrno · November 8, 2023, 6:29am

Would appreciate some points in the right direction here. I have a new stack up and running but Kibana is bouncing availalbe - degraded, frequently but not at regular intervals. The environment is new, (so am I to Elastic), all VM's, it's 1 Kibana 8.6.1 server, 1 fleet server, 5 esnodes. I run GET /_cat/indices all indexes are green and open, no obvious issues. get /_cluster/health is also green. Neither changes when Kibana goes degraded. I verified network is stable and good between all es nodes, fleet and kibana. No issues that I can find with memory, CPU, IO, swap, etc.

The load is what I consider small, 25 of our lab servers. I installed Agent, and then fleet to apply nginx, postgres, systtem and custom logs integrations. The 25 servers see very little load, 0-10 nginx connections per second.

What did I design/deploy wrong, or where should I look to see why Kibana is unhappy? Thanks in advance!

This is from kibana.log, nothing has been removed.

l":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"9ff8f522712bd4889b9139fe80bebe73"},"transaction":{"id":"8c35b05685fbd309"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:07:13.985-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"2d410fa8e494a998d603d2e1837f0b9b"},"transaction":{"id":"be9564416e76c041"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:08:31.576-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"8809bb5542125a60ecd0f43c70894366"},"transaction":{"id":"867bd7e7ed0d357f"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:08:37.582-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"bdedb8f09e6cdaff646ed2f70c2a9d00"},"transaction":{"id":"998dd007d0ff6045"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:09:19.404-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:10:19.403-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:10:27.603-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"23f4dd83cb9223882b091935d156f296"},"transaction":{"id":"11952a50677f8ab6"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:11:04.631-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"ce88946f967e5ec8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:11:48.328-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"65fbae9751b25ebb716de931c9ed77b6"},"transaction":{"id":"c1fb2b145e4654f7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:12:24.155-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"29205fc98e40e2c1bc45d3f5937b8f6b"},"transaction":{"id":"fd23faeb81fde16a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:12:36.273-06:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":28,\"healthy\":27,\"unhealthy\":1,\"offline\":0,\"inactive\":0,\"unenrolled\":2,\"total_all_statuses\":30,\"updating\":0},\"fleet_server\":{\"total_enrolled\":2,\"healthy\":1,\"unhealthy\":1,\"offline\":0,\"updating\":0,\"total_all_statuses\":2,\"num_host_urls\":1}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"2c3a5968f89954ec"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:13:07.952-06:00","message":"Logging in with provider \"basic\" (basic)","log":{"level":"INFO","logger":"plugins.security.routes"},"process":{"pid":81771},"trace":{"id":"6ccd6ddb2f72d47e4e17e096595af935"},"transaction":{"id":"b3a582ba83a0272e"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:13:09.824-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"86ada8b98c7199c4ff90aaf1d9d9e86e"},"transaction":{"id":"a6df4bfba89661e8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:13:19.404-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:13:25.508-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"e860080146010535338da107b81ef421"},"transaction":{"id":"3a5f4131eefe9359"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:13:31.635-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"f30ba8907c19c5463474a0588d2f3a26"},"transaction":{"id":"c0ff723505533b59"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:15:02.258-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"9a4a3b50ba0c7b7f296d42cb23efaa58"},"transaction":{"id":"c134ba4f61d8acee"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:15:08.267-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"32d9e3ef0ef7042900c94f0abe7c8bed"},"transaction":{"id":"2719c4f014975ff4"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:16:27.718-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"17363c5642e9edbc3890dd3e7de4fe35"},"transaction":{"id":"d237979730370438"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:16:48.918-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"731661859ff1c6290be9f7527111b5fb"},"transaction":{"id":"3d969cc7d322e0ab"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:17:10.531-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"98973ee2172eef0d705383d39517db2a"},"transaction":{"id":"4fa81b65c1fda95f"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T00:17:19.407-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}

carly.richmond · November 8, 2023, 10:14am

Hi @wrsnrno,

Welcome! To confirm are you finding availability issues with Kibana at all? Are there any errors in the kibana.log rather than INFO level messages?

If you share your Kibana config as well that could help too.

wrsnrno · November 8, 2023, 5:25pm

@carly.richmond Thank you for the quick reply! I'm not experiencing any availability issues with Kibana. My Kibaba use is limited to just manage Fleet and monitor logs being collected. I can "stream" logs while Kibana health is bouncing. In the Kibaba log I do see "service" events from time to time. Below are the most recent 100 entries in the kibana.log file. Is there a way to enable debug or higher level logging in Kibana for troubleshooting?

{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:41:45.407-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"cc0afec9195cacf2de9ace79fbc043a3"},"transaction":{"id":"4b3cb401a2257804"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:42:17.060-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"5945388da5a2af33eae40c49eb187084"},"transaction":{"id":"28feeda30cdfb9af"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:42:23.065-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"ff6d4c5f465a43db3b810cdecb8e1b6c"},"transaction":{"id":"1573684bcd59fd50"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:42:54.313-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"091055e76cf687aa1ba9021ca8c2f1b5"},"transaction":{"id":"384521153780f36d"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:43:00.322-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"d08a8d6f0d3238990f664beb80e68bdb"},"transaction":{"id":"192e3aca58091d4f"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:43:17.385-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"0b74077d5190ebd52045c5e7e0ac4917"},"transaction":{"id":"09d3d4eb9f8140ad"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:43:23.509-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"d92376b2c7b0b0df2d8eb399d6cb08d5"},"transaction":{"id":"bacd0d429dfd63be"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:43:57.026-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"70851f4989d28a87c5556d1f1a796178"},"transaction":{"id":"0d4414751a2dfd7f"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:44:25.557-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"c292d63034a0737932382510cf43d651"},"transaction":{"id":"3650ca723e2aa9ce"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:44:37.860-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"84ff38cafb3e1391288d93f02b8ceff8"},"transaction":{"id":"2895eb38b9830da0"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:44:41.877-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"26504ce416cab38914bc9f1d1536bc08"},"transaction":{"id":"c6127ce63845aee1"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:45:19.470-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:45:24.716-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"2e0f5417a2e7bd5f4b0e37b81175c293"},"transaction":{"id":"3a1f3858191b478b"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:47:05.356-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"0d8224b433798afc7e276f63d8a88e3b"},"transaction":{"id":"ca75c74159d3a92e"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:47:18.798-06:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":28,\"healthy\":27,\"unhealthy\":1,\"offline\":0,\"inactive\":0,\"unenrolled\":2,\"total_all_statuses\":30,\"updating\":0},\"fleet_server\":{\"total_enrolled\":2,\"healthy\":1,\"unhealthy\":1,\"offline\":0,\"updating\":0,\"total_all_statuses\":2,\"num_host_urls\":1}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"98316c3a52432061"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:47:24.898-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"56840ecbce2f792abedcd2bf0c62b822"},"transaction":{"id":"90dd4a018e4cb4e0"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:48:19.360-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"3234da03111bff6353363beacb614826"},"transaction":{"id":"8b2adbfbb248b72e"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:48:58.600-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"964b0a2d783919f0155530603900f372"},"transaction":{"id":"8bb5fbb71beaf840"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:50:39.931-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"ec51a658dee573a94bf0e90009aa9091"},"transaction":{"id":"7dc4e39f7ccc138b"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:50:45.936-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"6ca78b88243c260944c060617ae5d595"},"transaction":{"id":"b20d2f7b37a127fa"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:51:19.472-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:51:25.575-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"ae9a135af2959ac160d8089618003c96"},"transaction":{"id":"1b7bee888539ab80"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:52:10.055-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"746fe31e721ec6d14245a27ef2e086ec"},"transaction":{"id":"c02a66466894e94d"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:52:16.145-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"54dbaebd7064c08199d39c66f720fc79"},"transaction":{"id":"e316d9cf216ab012"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:52:33.644-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"b6552cb2a0a997c627e5708ca0241dc3"},"transaction":{"id":"ee57809e498fc600"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:52:39.651-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"b8d35b696fe07fe03cfcd7e96e652ee6"},"transaction":{"id":"f4cc982ddf023b8a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:52:55.443-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"247e22fae47254cd3fe4050e8c99804d"},"transaction":{"id":"c4da0cab55a67662"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:53:01.452-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"007731c5fe2364b22429c44fdefb14a5"},"transaction":{"id":"19230b44cdc9a6e3"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:53:19.472-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:53:54.352-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"edc843e889b82a48d67fd142b81406f5"},"transaction":{"id":"699f6cfe4ca65140"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:54:12.443-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"83c738c789ba90793abe5cb2bafb5915"},"transaction":{"id":"ccbe231856c9c928"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:55:01.657-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"37a6ffe69248b8608658600954e3c7bd"},"transaction":{"id":"5773cd72dd1656ec"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:55:50.345-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"0007031ffdf404e1280f987c88c5ae0b"},"transaction":{"id":"f07b1ece74f84128"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:56:10.945-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"a990d20f0837b11db5e2056b7ea4f106"},"transaction":{"id":"c117ab26e47d61c3"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:56:37.885-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"7c4eb08e07c3c82d2f0121cf5d64b71d"},"transaction":{"id":"e0eadf2a67d57430"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:56:43.892-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"701f276e0bd3c4fc197b9dc5cae3b80c"},"transaction":{"id":"684a143d0d457fb3"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:57:08.246-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"48a689744f18d03f65c1c66493a856e5"},"transaction":{"id":"68e3beb57080addb"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:57:19.472-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:57:23.694-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"bfea320b6e059ae3b67bf1fc0b7edd15"},"transaction":{"id":"461872c5e558e118"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:57:29.700-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"a2a0b0831df21d68ea8834e3e2103d26"},"transaction":{"id":"4caee6808ee8bd32"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:57:57.192-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"002bc3d60729357a4e78e292e0ced574"},"transaction":{"id":"806ac27ee75e1e14"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:58:17.819-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"e3d8ee65cef6969f0857e4bccd108622"},"transaction":{"id":"dc9115c70f4c52cc"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:59:17.263-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"352d442749ac47ed5cfb959cdd156449"},"transaction":{"id":"b3d8b88b04d7dd44"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:59:39.747-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"7f91172c35e6f798591c220e13747b59"},"transaction":{"id":"1bfdbc9d81e53146"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:59:52.693-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"ddc151b241120c36"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T10:59:58.702-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"5402b66e1fc387a98c1b7e5e922b80fa"},"transaction":{"id":"01603ebf3fafe167"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:01:12.708-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"65b1372e3e30cba0ec17439d69fa16cc"},"transaction":{"id":"b6d35bbce25aba20"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:01:18.715-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"f1541820bcc47ccdbfb201a7f9941275"},"transaction":{"id":"b08b3ab7985f4c11"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:02:28.021-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"aa46ac415d4533aa3d62e497c367c926"},"transaction":{"id":"027c7a9a7d07bb22"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:02:28.071-06:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":28,\"healthy\":27,\"unhealthy\":1,\"offline\":0,\"inactive\":0,\"unenrolled\":2,\"total_all_statuses\":30,\"updating\":0},\"fleet_server\":{\"total_enrolled\":2,\"healthy\":1,\"unhealthy\":1,\"offline\":0,\"updating\":0,\"total_all_statuses\":2,\"num_host_urls\":1}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"1c18a00c26f3e6f4"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:02:34.030-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"4023936eedd4dd0710187e0fe41beee3"},"transaction":{"id":"c8ff1a51776233ca"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:02:51.498-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"7459ced5d2f1d0315de2c24e3a487bcf"},"transaction":{"id":"50e2e86387a766fd"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:03:02.388-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"15a13171532f39c2da231abab73bf5f3"},"transaction":{"id":"185c8232bb53642d"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:03:14.528-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"32e773a3ab3580a40523bb57b2acb057"},"transaction":{"id":"cbc9a9780725530a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:03:19.473-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:04:05.635-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"b93542870bafb4d7efea2dee14e36fae"},"transaction":{"id":"0058c340461a7200"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:05:19.473-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:05:25.469-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"db6e61cc625edf9fbc3fba81b4962e2f"},"transaction":{"id":"2774caadbae4fad2"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:05:31.535-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"e2017fefab96f054ffe7569ff233f639"},"transaction":{"id":"ff426eac843c32ea"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:05:48.664-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"121dc7cb4207f8436e390b80d823a304"},"transaction":{"id":"30db211d36d44173"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:05:54.670-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"685724f57995c7031352660c54d9ac78"},"transaction":{"id":"784826353f06c61f"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:06:19.474-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:06:24.523-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"7e0bfd02db78d4f9fecc9c6726fe1c4b"},"transaction":{"id":"653dd59a2a627e52"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:06:38.683-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"1407d905fc4c9ef4f260c5d1e12284aa"},"transaction":{"id":"dfc538678b9d33ea"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:06:44.749-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"ee9450789b3670516cb1df699fec792c"},"transaction":{"id":"b1c47aea69856a67"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:08:38.184-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"35c1462382e8395e5059f400203e4c68"},"transaction":{"id":"c83f947918e497ed"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:08:44.198-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"d0164918c48c83f85e277f3f6c841fc9"},"transaction":{"id":"d9a13c07c7a94d61"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:10:30.198-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"dfc2b477886cd89d2d91fd1421864a35"},"transaction":{"id":"d92c515886f41d85"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:10:36.249-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"30399d063c7d8580f5b1b459b41d1bd7"},"transaction":{"id":"2cb894eea8bc8c7a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:11:03.185-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"04a5ef425367ff5f249a09e6d85be540"},"transaction":{"id":"348029d73cc6fc7c"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:11:23.458-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"782b767701b1beaa6047815f90ceb66c"},"transaction":{"id":"92c4bc97b0c7da88"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:12:15.192-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"d1e3e1260f9443b3666c1affec98656f"},"transaction":{"id":"ce7356c9f1c4ef61"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:13:19.476-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:13:27.166-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"d2eff38db2ef898397bde6524dd46019"},"transaction":{"id":"39c7a908d1e0f8fd"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:13:37.602-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"934da25e4c96ecb57d90e88c0b3fb08b"},"transaction":{"id":"666914cf51ae4a7e"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:13:53.108-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"6206b361ab8d4fdb4e7f9a1192ab4921"},"transaction":{"id":"c83381d80d0ece69"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:14:16.180-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"1c11badb5a8c99553c5b3ee1cbff6380"},"transaction":{"id":"49555192996e41de"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:15:10.529-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"ff3081409dcbd133c34ecd83eb544da3"},"transaction":{"id":"4e89b800d090107a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:15:16.534-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fdfc945fa658759099e3acc9c2f993e7"},"transaction":{"id":"df3356c5cb4c0d11"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:15:52.487-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"f8d79e7956a160de53bdcbc56860d437"},"transaction":{"id":"d0324755ce8d9553"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:15:58.496-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"9ddd490643cb7d07435b99d3b635d1ea"},"transaction":{"id":"3d23d52f7247ab5a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:16:29.654-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"efb84323aca5da7e45a8edb699ba3e1d"},"transaction":{"id":"6bd128e04dd0b704"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:16:40.836-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"5f881e621472a3cd"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:16:57.487-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"9f70ae64530b4738f940e4a2f18f4bbf"},"transaction":{"id":"21105e1dd8b96235"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:17:03.593-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"e383b5ab8e4a65a2d963a99382046528"},"transaction":{"id":"e0b2fb8f1314e3c2"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:17:30.688-06:00","message":"Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":28,\"healthy\":27,\"unhealthy\":1,\"offline\":0,\"inactive\":0,\"unenrolled\":2,\"total_all_statuses\":30,\"updating\":0},\"fleet_server\":{\"total_enrolled\":2,\"healthy\":1,\"unhealthy\":1,\"offline\":0,\"updating\":0,\"total_all_statuses\":2,\"num_host_urls\":1}}","log":{"level":"INFO","logger":"plugins.fleet"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"d5a13653dad5e0e7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:18:04.600-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"363570b4c28fe86f0c0d1f96cff33c28"},"transaction":{"id":"671e5a5bf6150f14"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:18:43.031-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"1a21ef539011401bf5ca3847d0ec449f"},"transaction":{"id":"ebdb9b8dad4ef54e"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:19:32.440-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"594d8c96e7cfd4006d30fa8de92c30e3"},"transaction":{"id":"b7cf8f1f8164c136"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:19:56.806-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"dce3d79d1e093bb08cf70a93927095b5"},"transaction":{"id":"231a13256e9059a0"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:20:28.927-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"d0d8df608262184e0e3b2d093ac74e4c"},"transaction":{"id":"4f1647bd4acf8bb9"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:20:35.033-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"4afd3750f53ecccb43310b7f80939999"},"transaction":{"id":"964e438551fe7029"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:21:26.682-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"6e4a6f84c113cb1c907d81da6b2c976a"},"transaction":{"id":"c4c1826ec6397677"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:21:32.691-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"b25b9099ccc44ee5c98247a3d44d56f5"},"transaction":{"id":"d3d8261265ee3659"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:21:59.030-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"13abcbe4f33c9d8334397f9b17cbd0c5"},"transaction":{"id":"faa97c85e1e161a5"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:22:51.955-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"80cca37bd9c936dddf65763045249158"},"transaction":{"id":"90c0fa11bee8c767"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:23:14.017-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"c1e5ff3fc96a51978e20c3706228b073"},"transaction":{"id":"204098ed47f28cda"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:23:19.478-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"fa90dc3d97b3fe322b675df8e239b2de"},"transaction":{"id":"445c3fd80611ad97"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:23:28.717-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"1dc6283b8460d825"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-08T11:23:34.725-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":81771},"trace":{"id":"2e52e0c94d5e372a613205808bf9337f"},"transaction":{"id":"6832780e3edf4a46"}}

carly.richmond · November 8, 2023, 5:42pm

Yes, you can change the default logging level as covered in this example.

If you want to dive more into the logging configuration for Kibana I would also recommend reading about the logging settings here.

wrsnrno · November 10, 2023, 1:06am

@carly.richmond - Thanks for the logging help, finally had the opportunity to sift through the results. Below is the output of tail -f kibana.log | grep -v 'ignore_unavailable' | grep -E 'degraded|available' There is a pattern showing 114 services are degraded and then all services are available.

My question is, where do I find See the status page for more information as mentioned in the output ? Its not clear. How do I diagnose this?

I also found this post about task manager. When I ran the curl command from the Dev Tools console, it returns 400. Because the diagnosis does not work, perhaps this is not going in the correct direction.

{"kibana":{"status":{"level":"degraded","summary":"114 services are degraded: security, cloudLinks, data and 111 other(s)","detail":"See the status page for more information","meta":{"affectedServices":["security","cloudLinks","data","encryptedSavedObjects","files","lists","snapshotRestore","telemetry","actions","dataViewEditor","dataViewFieldEditor","ecsDataQualityDashboard","fileUpload","filesManagement","licenseManagement","savedObjects","telemetryManagementSection","ingestPipelines","notifications","savedObjectsTaggingOss","watcher","savedObjectsManagement","savedObjectsTagging","embeddable","globalSearchBar","unifiedSearch","dataViewManagement","imageEmbeddable","navigation","presentationUtil","savedSearch","uiActionsEnhanced","controls","embeddableEnhanced","expressionError","expressionImage","expressionMetric","expressionRepeatImage","expressionRevealImage","expressionShape","graph","kibanaOverview","urlDrilldown","visualizations","dashboard","eventAnnotation","expressionGauge","expressionHeatmap","expressionLegacyMetricVis","expressionMetricVis","expressionPartitionVis","expressionTagcloud","visDefaultEditor","visTypeHeatmap","visTypeMarkdown","visTypeMetric","visTypeTable","visTypeTagcloud","visTypeTimelion","visTypeTimeseries","visTypeVega","visTypeVislib","visTypeXy","dashboardEnhanced","expressionXY","inputControlVis","triggersActionsUi","visTypeGauge","visTypePie","lens","ruleRegistry","stackAlerts","stackConnectors","transform","cases","discover","maps","aiops","dataVisualizer","discoverEnhanced","observabilityShared","reporting","threatIntelligence","timelines","canvas","cloudSecurityPosture","discoverLogExplorer","indexManagement","ml","observabilityAIAssistant","osquery","sessionView","exploratoryView","indexLifecycleManagement","kubernetesSecurity","logsShared","remoteClusters","rollup","cloudDefend","crossClusterReplication","enterpriseSearch","observability","infra","observabilityOnboarding","securitySolution","synthetics","uptime","apm","monitoring","securitySolutionEss","upgradeAssistant","logstash","ux","taskManager"]}}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:20.839-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"1f59ac12af3d26189b69c580c75ddedf"},"transaction":{"id":"b81674e87f181447"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:20.840-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"1f59ac12af3d26189b69c580c75ddedf"},"transaction":{"id":"b81674e87f181447"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:31.559-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"383e5153c048d6786484e507778c05a5"},"transaction":{"id":"6c7e32ee581e59de"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:31.624-06:00","message":"Last computed manifest not available yet","log":{"level":"DEBUG","logger":"plugins.securitySolution.endpoint:user-artifact-packager:1.0.0"},"process":{"pid":87728},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"26f539639bbe1549"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:37.568-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"3dc360c7e6a8a0a0ca1baa35f80cb672"},"transaction":{"id":"f6bebbbeff2b95db"}}
{"kibana":{"status":{"level":"available","summary":"All services are available"}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:37.649-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"3dc360c7e6a8a0a0ca1baa35f80cb672"},"transaction":{"id":"f6bebbbeff2b95db"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:37.650-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"3dc360c7e6a8a0a0ca1baa35f80cb672"},"transaction":{"id":"f6bebbbeff2b95db"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:43.567-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"d88e66e41928ca36f6d0ad4bcba8f063"},"transaction":{"id":"8482296e1fa093c3"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:49:48.443-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"0160ffc99c6072a361da951d03a3a6ac"},"transaction":{"id":"8af411b1cad69fa8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:02.644-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"cedf7d9e6fb9cc0a676ef3b2113f0bc2"},"transaction":{"id":"d94c684434d95863"}}
{"kibana":{"status":{"level":"degraded","summary":"114 services are degraded: security, cloudLinks, data and 111 other(s)","detail":"See the status page for more information","meta":{"affectedServices":["security","cloudLinks","data","encryptedSavedObjects","files","lists","snapshotRestore","telemetry","actions","dataViewEditor","dataViewFieldEditor","ecsDataQualityDashboard","fileUpload","filesManagement","licenseManagement","savedObjects","telemetryManagementSection","ingestPipelines","notifications","savedObjectsTaggingOss","watcher","savedObjectsManagement","savedObjectsTagging","embeddable","globalSearchBar","unifiedSearch","dataViewManagement","imageEmbeddable","navigation","presentationUtil","savedSearch","uiActionsEnhanced","controls","embeddableEnhanced","expressionError","expressionImage","expressionMetric","expressionRepeatImage","expressionRevealImage","expressionShape","graph","kibanaOverview","urlDrilldown","visualizations","dashboard","eventAnnotation","expressionGauge","expressionHeatmap","expressionLegacyMetricVis","expressionMetricVis","expressionPartitionVis","expressionTagcloud","visDefaultEditor","visTypeHeatmap","visTypeMarkdown","visTypeMetric","visTypeTable","visTypeTagcloud","visTypeTimelion","visTypeTimeseries","visTypeVega","visTypeVislib","visTypeXy","dashboardEnhanced","expressionXY","inputControlVis","triggersActionsUi","visTypeGauge","visTypePie","lens","ruleRegistry","stackAlerts","stackConnectors","transform","cases","discover","maps","aiops","dataVisualizer","discoverEnhanced","observabilityShared","reporting","threatIntelligence","timelines","canvas","cloudSecurityPosture","discoverLogExplorer","indexManagement","ml","observabilityAIAssistant","osquery","sessionView","exploratoryView","indexLifecycleManagement","kubernetesSecurity","logsShared","remoteClusters","rollup","cloudDefend","crossClusterReplication","enterpriseSearch","observability","infra","observabilityOnboarding","securitySolution","synthetics","uptime","apm","monitoring","securitySolutionEss","upgradeAssistant","logstash","ux","taskManager"]}}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:02.726-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"cedf7d9e6fb9cc0a676ef3b2113f0bc2"},"transaction":{"id":"d94c684434d95863"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:02.726-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"cedf7d9e6fb9cc0a676ef3b2113f0bc2"},"transaction":{"id":"d94c684434d95863"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:15.366-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"54b2456752f7281596700c677fe23066"},"transaction":{"id":"479a32ad40a14b96"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:27.440-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"58293c49d26c14f8ccc629b843d8724f"},"transaction":{"id":"9bb22b156746c22c"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:41.497-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"c81ee2e5d538c1a4807768e6a166770a"},"transaction":{"id":"411169a39539c9b0"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:48.446-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"0160ffc99c6072a361da951d03a3a6ac"},"transaction":{"id":"8af411b1cad69fa8"}}
{"kibana":{"status":{"level":"available","summary":"All services are available"}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:48.528-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"0160ffc99c6072a361da951d03a3a6ac"},"transaction":{"id":"8af411b1cad69fa8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:48.530-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"0160ffc99c6072a361da951d03a3a6ac"},"transaction":{"id":"8af411b1cad69fa8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:52.841-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"24c19fc6c894a90875887fe250e69786"},"transaction":{"id":"152b0f61714fadeb"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:50:52.845-06:00","message":"Last computed manifest not available yet","log":{"level":"DEBUG","logger":"plugins.securitySolution.endpoint:user-artifact-packager:1.0.0"},"process":{"pid":87728},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"3e34e7c7c2a003aa"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:05.886-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"bbcfd1ee617f157757dd147ee07efb70"},"transaction":{"id":"fdcd5a2bdb644826"}}
{"kibana":{"status":{"level":"degraded","summary":"114 services are degraded: security, cloudLinks, data and 111 other(s)","detail":"See the status page for more information","meta":{"affectedServices":["security","cloudLinks","data","encryptedSavedObjects","files","lists","snapshotRestore","telemetry","actions","dataViewEditor","dataViewFieldEditor","ecsDataQualityDashboard","fileUpload","filesManagement","licenseManagement","savedObjects","telemetryManagementSection","ingestPipelines","notifications","savedObjectsTaggingOss","watcher","savedObjectsManagement","savedObjectsTagging","embeddable","globalSearchBar","unifiedSearch","dataViewManagement","imageEmbeddable","navigation","presentationUtil","savedSearch","uiActionsEnhanced","controls","embeddableEnhanced","expressionError","expressionImage","expressionMetric","expressionRepeatImage","expressionRevealImage","expressionShape","graph","kibanaOverview","urlDrilldown","visualizations","dashboard","eventAnnotation","expressionGauge","expressionHeatmap","expressionLegacyMetricVis","expressionMetricVis","expressionPartitionVis","expressionTagcloud","visDefaultEditor","visTypeHeatmap","visTypeMarkdown","visTypeMetric","visTypeTable","visTypeTagcloud","visTypeTimelion","visTypeTimeseries","visTypeVega","visTypeVislib","visTypeXy","dashboardEnhanced","expressionXY","inputControlVis","triggersActionsUi","visTypeGauge","visTypePie","lens","ruleRegistry","stackAlerts","stackConnectors","transform","cases","discover","maps","aiops","dataVisualizer","discoverEnhanced","observabilityShared","reporting","threatIntelligence","timelines","canvas","cloudSecurityPosture","discoverLogExplorer","indexManagement","ml","observabilityAIAssistant","osquery","sessionView","exploratoryView","indexLifecycleManagement","kubernetesSecurity","logsShared","remoteClusters","rollup","cloudDefend","crossClusterReplication","enterpriseSearch","observability","infra","observabilityOnboarding","securitySolution","synthetics","uptime","apm","monitoring","securitySolutionEss","upgradeAssistant","logstash","ux","taskManager"]}}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:05.967-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"bbcfd1ee617f157757dd147ee07efb70"},"transaction":{"id":"fdcd5a2bdb644826"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:05.968-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"bbcfd1ee617f157757dd147ee07efb70"},"transaction":{"id":"fdcd5a2bdb644826"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:17.613-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"ecfdba31069d4e12946068e1609d3ea0"},"transaction":{"id":"7854e3b9aa5eea9f"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:23.642-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"51ec4d1c4a7bb26ac9aee47842fe0915"},"transaction":{"id":"dc74c9c9d0604e82"}}
{"kibana":{"status":{"level":"available","summary":"All services are available"}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:23.723-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"51ec4d1c4a7bb26ac9aee47842fe0915"},"transaction":{"id":"dc74c9c9d0604e82"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:23.724-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"51ec4d1c4a7bb26ac9aee47842fe0915"},"transaction":{"id":"dc74c9c9d0604e82"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:29.646-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"6cd3aaf2cff5c384a4c64b42f2f939a2"},"transaction":{"id":"f21e64296a732d3b"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:35.684-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"5061c854344d14614a51487ad24b6570"},"transaction":{"id":"f731501c84ad3775"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:41.624-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"a62e36ae3be9eb620ac8abb17da938f9"},"transaction":{"id":"40eb1537ed65061d"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:47.624-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"0ad24819dafe7531e46e05d7db342d7e"},"transaction":{"id":"dc210c591ceb6ce1"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:53.640-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"dcfcb97ff57e098624803e152391c6fb"},"transaction":{"id":"7d0b30d13498e69a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:51:53.702-06:00","message":"Last computed manifest not available yet","log":{"level":"DEBUG","logger":"plugins.securitySolution.endpoint:user-artifact-packager:1.0.0"},"process":{"pid":87728},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"26ab9af948741aff"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:05.092-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"bff16b4c4693fba3e179b467467b2744"},"transaction":{"id":"4eee98061cc68657"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:17.428-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"ee63f96a7ad19a99"}}
{"kibana":{"status":{"level":"degraded","summary":"114 services are degraded: security, cloudLinks, data and 111 other(s)","detail":"See the status page for more information","meta":{"affectedServices":["security","cloudLinks","data","encryptedSavedObjects","files","lists","snapshotRestore","telemetry","actions","dataViewEditor","dataViewFieldEditor","ecsDataQualityDashboard","fileUpload","filesManagement","licenseManagement","savedObjects","telemetryManagementSection","ingestPipelines","notifications","savedObjectsTaggingOss","watcher","savedObjectsManagement","savedObjectsTagging","embeddable","globalSearchBar","unifiedSearch","dataViewManagement","imageEmbeddable","navigation","presentationUtil","savedSearch","uiActionsEnhanced","controls","embeddableEnhanced","expressionError","expressionImage","expressionMetric","expressionRepeatImage","expressionRevealImage","expressionShape","graph","kibanaOverview","urlDrilldown","visualizations","dashboard","eventAnnotation","expressionGauge","expressionHeatmap","expressionLegacyMetricVis","expressionMetricVis","expressionPartitionVis","expressionTagcloud","visDefaultEditor","visTypeHeatmap","visTypeMarkdown","visTypeMetric","visTypeTable","visTypeTagcloud","visTypeTimelion","visTypeTimeseries","visTypeVega","visTypeVislib","visTypeXy","dashboardEnhanced","expressionXY","inputControlVis","triggersActionsUi","visTypeGauge","visTypePie","lens","ruleRegistry","stackAlerts","stackConnectors","transform","cases","discover","maps","aiops","dataVisualizer","discoverEnhanced","observabilityShared","reporting","threatIntelligence","timelines","canvas","cloudSecurityPosture","discoverLogExplorer","indexManagement","ml","observabilityAIAssistant","osquery","sessionView","exploratoryView","indexLifecycleManagement","kubernetesSecurity","logsShared","remoteClusters","rollup","cloudDefend","crossClusterReplication","enterpriseSearch","observability","infra","observabilityOnboarding","securitySolution","synthetics","uptime","apm","monitoring","securitySolutionEss","upgradeAssistant","logstash","ux","taskManager"]}}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:17.509-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"ee63f96a7ad19a99"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:17.509-06:00","message":"Kibana is now degraded (was available)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"ee63f96a7ad19a99"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:33.533-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"f55e59dd2492a3b68c6334df72371da8"},"transaction":{"id":"ddf579d1f7ea2fa2"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:48.443-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"0160ffc99c6072a361da951d03a3a6ac"},"transaction":{"id":"8af411b1cad69fa8"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:54.461-06:00","message":"status serviceStatus now set to available","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"589df1dce91571ebcdfbf9a6b5d13951"},"transaction":{"id":"554530101a4fc199"}}
{"kibana":{"status":{"level":"available","summary":"All services are available"}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:54.542-06:00","message":"Recalculated overall status","log":{"level":"DEBUG","logger":"status"},"process":{"pid":87728},"trace":{"id":"589df1dce91571ebcdfbf9a6b5d13951"},"transaction":{"id":"554530101a4fc199"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:52:54.543-06:00","message":"Kibana is now available (was degraded)","log":{"level":"INFO","logger":"status"},"process":{"pid":87728},"trace":{"id":"589df1dce91571ebcdfbf9a6b5d13951"},"transaction":{"id":"554530101a4fc199"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2023-11-09T18:53:04.853-06:00","message":"status serviceStatus now set to degraded","log":{"level":"DEBUG","logger":"plugins.taskManager"},"process":{"pid":87728},"trace":{"id":"07a0ee80135431717aebbb4b1d312774"},"transaction":{"id":"f8f14d849bf226c3"}}
{"kibana":{"status":{"level":"degraded","summary":"114 services are degraded: security, cloudLinks, data and 111 other(s)","detail":"See the status page for more information","meta":{"affectedServices":["security","cloudLinks","data","encryptedSavedObjects","files","lists","snapshotRestore","telemetry","actions","dataViewEditor","dataViewFieldEditor","ecsDataQualityDashboard","fileUpload","filesManagement","licenseManagement","savedObjects","telemetryManagementSection","ingestPipelines","notifications","savedObjectsTaggingOss","watcher","savedObjectsManagement","savedObjectsTagging","embeddable","globalSearchBar","unifiedSearch","dataViewManagement","imageEmbeddable","navigation","presentationUtil","savedSearch","uiActionsEnhanced","controls","embeddableEnhanced","expressionError","expressionImage","expressionMetric","expressionRepeatImage","expressionRevealImage","expressionShape","graph","kibanaOverview","urlDrilldown","visualizations","dashboard","eventAnnotation","expressionGauge","expressionHeatmap","expressionLegacyMetricVis","expressionMetricVis","expressionPartitionVis","expressionTagcloud","visDefaultEditor","visTypeHeatmap","visTypeMarkdown","visTypeMetric","visTypeTable","visTypeTagcloud","visTypeTimelion","visTypeTimeseries","visTypeVega","visTypeVislib","visTypeXy","dashboardEnhanced","expressionXY","inputControlVis","triggersActionsUi","visTypeGauge","visTypePie","lens","ruleRegistry","stackAlerts","stackConnectors","transform","cases","discover","maps","aiops","dataVisualizer","discoverEnhanced","observabilityShared","reporting","threatIntelligence","timelines","canvas","cloudSecurityPosture","discoverLogExplorer","indexManagement","ml","observabilityAIAssistant","osquery","sessionView","exploratoryView","indexLifecycleManagement","kubernetesSecurity","logsShared","remoteClusters","rollup","cloudDefend","crossClusterReplication","enterpriseSearch","observability","infra","observabilityOnboarding","securitySolution","synthetics","uptime","apm","monitoring","securitySolutionEss","upgradeAssistant","logstash","ux","taskManager"]}}},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@time

system · December 8, 2023, 1:07am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.