Service unavailable error code 503 all shard failed

Elastic Stack Elasticsearch
1

Hello all,
I got this problem showing that service unavailable

{"statusCode":503,"error":"Service Unavailable","message":"[all shards failed: search_phase_execution_exception\n\tRoot causes:\n\t\tno_shard_available_action_exception: null]: all shards failed"}

i am new with this kinda stuff. I got this problem after refreshing discover tab
this is what i get when using get _cat/shards?v

.ds-logs-elastic_agent.metricbeat-default-2023.05.14-000005        0     p      STARTED       52805  13.2mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.14-000005        0     r      UNASSIGNED                            
.async-search                                                      0     p      STARTED           0     4kb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.17-000007              0     p      STARTED         259 463.9kb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.17-000007              0     r      UNASSIGNED                            
.ds-logs-elastic_agent.filebeat-default-2023.05.14-000005          0     p      STARTED       21642   7.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.14-000005          0     r      UNASSIGNED                            
.ds-logs-endpoint.events.library-default-2023.05.15-000005         0     p      STARTED        3845   1.9mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.library-default-2023.05.15-000005         0     r      UNASSIGNED                            
.kibana-event-log-8.7.1-000001                                     0     p      STARTED    29823516   4.6gb 127.0.0.1 xenpher
.ds-metrics-system.process-default-2023.05.04-000001               0     p      STARTED     3390526   2.3gb 127.0.0.1 xenpher
.ds-metrics-system.process-default-2023.05.04-000001               0     r      UNASSIGNED                            
.ds-logs-endpoint.events.registry-default-2023.05.13-000003        0     p      STARTED       29906     6mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.registry-default-2023.05.13-000003        0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.19-000010 0     p      STARTED       20763   4.4mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.19-000010 0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.19-000009         0     p      STARTED     1993780   850mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.19-000009         0     r      UNASSIGNED                            
.ds-logs-endpoint.alerts-default-2023.05.19-000004                 0     p      STARTED           0    225b 127.0.0.1 xenpher
.ds-logs-endpoint.alerts-default-2023.05.19-000004                 0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.14-000005 0     p      STARTED       48967   7.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.14-000005 0     r      UNASSIGNED                            
.ds-logs-endpoint.alerts-default-2023.05.12-000002                 0     p      STARTED          20 214.7kb 127.0.0.1 xenpher
.ds-logs-endpoint.alerts-default-2023.05.12-000002                 0     r      UNASSIGNED                            
filebeat-1                                                         0     p      STARTED           0    225b 127.0.0.1 xenpher
filebeat-1                                                         0     r      UNASSIGNED                            
.fleet-file-data-agent-000001                                      0     p      STARTED           0    225b 127.0.0.1 xenpher
.ds-metrics-system.load-default-2023.05.04-000001                  0     p      STARTED      289403  65.2mb 127.0.0.1 xenpher
.ds-metrics-system.load-default-2023.05.04-000001                  0     r      UNASSIGNED                            
.lists-default-000001                                              0     p      STARTED           0    225b 127.0.0.1 xenpher
.lists-default-000001                                              0     r      UNASSIGNED                            
.ds-logs-elastic_agent.fleet_server-default-2023.05.12-000003      0     p      STARTED         899   1.6mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.12-000003      0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.16-000007 0     p      STARTED       49202     7mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.16-000007 0     r      UNASSIGNED                            
.ds-logs-system.application-default-2023.05.13-000003              0     p      STARTED          67 164.1kb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.13-000003              0     r      UNASSIGNED                            
.fleet-files-endpoint-000001                                       0     p      STARTED           0    225b 127.0.0.1 xenpher
.ds-metrics-elastic_agent.filebeat-default-2023.05.04-000001       0     p      STARTED      817728 173.5mb 127.0.0.1 xenpher
.ds-metrics-elastic_agent.filebeat-default-2023.05.04-000001       0     r      UNASSIGNED                            
.security-7                                                        0     p      STARTED         921   603kb 127.0.0.1 xenpher
.apm-custom-link                                                   0     p      STARTED           0    225b 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.14-000004            0     p      STARTED      195564  34.6mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.14-000004            0     r      UNASSIGNED                            
.ds-metrics-endpoint.metrics-default-2023.05.04-000001             0     p      STARTED         123 986.7kb 127.0.0.1 xenpher
.ds-metrics-endpoint.metrics-default-2023.05.04-000001             0     r      UNASSIGNED                            
.apm-source-map                                                    0     p      STARTED           0    225b 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.16-000007      0     p      STARTED         449 360.7kb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.16-000007      0     r      UNASSIGNED                            
.ds-metrics-system.process.summary-default-2023.05.04-000001       0     p      STARTED      413360  82.2mb 127.0.0.1 xenpher
.ds-metrics-system.process.summary-default-2023.05.04-000001       0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.22-000013         0     p      STARTED     3974605   1.6gb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.22-000013         0     r      UNASSIGNED                            
.ds-metrics-system.network-default-2023.05.04-000001               0     p      STARTED     1395798 207.3mb 127.0.0.1 xenpher
.ds-metrics-system.network-default-2023.05.04-000001               0     r      UNASSIGNED                            
.ds-logs-endpoint.events.network-default-2023.05.04-000001         0     p      STARTED      310644 100.7mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.network-default-2023.05.04-000001         0     r      UNASSIGNED                            
.ds-logs-system.system-default-2023.05.12-000002                   0     p      STARTED          57 304.2kb 127.0.0.1 xenpher
.ds-logs-system.system-default-2023.05.12-000002                   0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.13-000004 0     p      STARTED       47040   5.1mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.13-000004 0     r      UNASSIGNED                            
.ds-logs-elastic_agent.fleet_server-default-2023.05.14-000005      0     p      STARTED         901   1.6mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.14-000005      0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.13-000003        0     p      STARTED         640 590.4kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.13-000003        0     r      UNASSIGNED                            
.ds-logs-system.application-default-2023.05.15-000005              0     p      STARTED         152 486.5kb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.15-000005              0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.19-000008        0     p      STARTED         572 464.8kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.19-000008        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.12-000002            0     p      STARTED       97677  22.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.12-000002            0     r      UNASSIGNED                            
.ds-logs-endpoint.events.api-default-2023.05.12-000002             0     p      STARTED           4  57.7kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.api-default-2023.05.12-000002             0     r      UNASSIGNED                            
.ds-logs-system.security-default-2023.05.16-000006                 0     p      STARTED        4109   1.5mb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.16-000006                 0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.04-000001                   0     p      STARTED        1638 639.7kb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.04-000001                   0     r      UNASSIGNED                            
.ds-logs-elastic_agent.filebeat-default-2023.05.17-000008          0     p      STARTED       90329    20mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.17-000008          0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.13-000003            0     p      STARTED       97494  21.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.13-000003            0     r      UNASSIGNED                            
.internal.alerts-security.alerts-default-000001                    0     p      STARTED        6108  26.8mb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.12-000002                     0     p      STARTED          97 221.6kb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.12-000002                     0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.04-000001        0     p      STARTED        5713   1.3mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.04-000001        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.library-default-2023.05.13-000003         0     p      STARTED        7828   2.4mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.library-default-2023.05.13-000003         0     r      UNASSIGNED                            
.ds-logs-elastic_agent.fleet_server-default-2023.05.05-000002      0     p      STARTED        4784   7.5mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.05-000002      0     r      UNASSIGNED                            
.ds-logs-elastic_agent.fleet_server-default-2023.05.17-000008      0     p      STARTED        3884   6.7mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.17-000008      0     r      UNASSIGNED                            
.ds-metrics-system.cpu-default-2023.05.04-000001                   0     p      STARTED      413442  78.2mb 127.0.0.1 xenpher
.ds-metrics-system.cpu-default-2023.05.04-000001                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.registry-default-2023.05.14-000004        0     p      STARTED       89224  14.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.registry-default-2023.05.14-000004        0     r      UNASSIGNED                            
.ds-metrics-system.diskio-default-2023.05.04-000001                0     p      STARTED     6183433 914.5mb 127.0.0.1 xenpher
.ds-metrics-system.diskio-default-2023.05.04-000001                0     r      UNASSIGNED                            
.ds-logs-system.application-default-2023.05.04-000001              0     p      STARTED        8187   2.6mb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.04-000001              0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.14-000005                   0     p      STARTED         273 374.8kb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.14-000005                   0     r      UNASSIGNED                            
.ds-metrics-system.socket_summary-default-2023.05.04-000001        0     p      STARTED      413415  96.5mb 127.0.0.1 xenpher
.ds-metrics-system.socket_summary-default-2023.05.04-000001        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.api-default-2023.05.16-000004             0     p      STARTED           4  57.7kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.api-default-2023.05.16-000004             0     r      UNASSIGNED                            
.ds-logs-system.auth-default-2023.05.13-000003                     0     p      STARTED          98 284.7kb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.13-000003                     0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.18-000009 0     p      STARTED       16242   3.2mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.18-000009 0     r      UNASSIGNED                            
.kibana_security_session_1                                         0     p      STARTED           1   6.6kb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.05-000002 0     p      STARTED      259405  26.8mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.05-000002 0     r      UNASSIGNED                            
.ds-logs-endpoint.events.library-default-2023.05.04-000001         0     p      STARTED       81395  15.5mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.library-default-2023.05.04-000001         0     r      UNASSIGNED                            
.ds-logs-endpoint.events.registry-default-2023.05.15-000005        0     p      STARTED       49085     8mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.registry-default-2023.05.15-000005        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.registry-default-2023.05.04-000001        0     p      STARTED      458053  67.3mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.registry-default-2023.05.04-000001        0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.12-000003                   0     p      STARTED         144 303.8kb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.12-000003                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.network-default-2023.05.16-000006         0     p      STARTED       14427   4.4mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.network-default-2023.05.16-000006         0     r      UNASSIGNED                            
.ds-logs-system.syslog-default-2023.05.17-000007                   0     p      STARTED     4361071 707.6mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.17-000007                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.20-000010         0     p      STARTED       47276  20.9mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.20-000010         0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.13-000004                   0     p      STARTED         143 297.8kb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.13-000004                   0     r      UNASSIGNED                            
.ds-logs-elastic_agent.fleet_server-default-2023.05.15-000006      0     p      STARTED        1001   1.7mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.15-000006      0     r      UNASSIGNED                            
.ds-logs-system.syslog-default-2023.05.13-000003                   0     p      STARTED      460835  93.8mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.13-000003                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.20-000010            0     p      STARTED       16317   4.1mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.20-000010            0     r      UNASSIGNED                            
.ds-logs-elastic_agent.metricbeat-default-2023.05.13-000004        0     p      STARTED       52629  13.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.13-000004        0     r      UNASSIGNED                            
.transform-notifications-000002                                    0     p      STARTED         151 108.4kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.20-000009        0     p      STARTED         220 221.2kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.20-000009        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.17-000007            0     p      STARTED      449251  97.1mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.17-000007            0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.04-000001            0     p      STARTED     2226402 457.7mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.04-000001            0     r      UNASSIGNED                            
metrics-endpoint.metadata_current_default                          0     p      STARTED          11  90.4kb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.04-000001                     0     p      STARTED        2807   1.4mb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.04-000001                     0     r      UNASSIGNED                            
.ds-logs-endpoint.events.network-default-2023.05.14-000004         0     p      STARTED       60340  15.5mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.network-default-2023.05.14-000004         0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.14-000004        0     p      STARTED        1212 952.8kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.14-000004        0     r      UNASSIGNED                            
.ds-logs-elastic_agent.metricbeat-default-2023.05.04-000001        0     p      STARTED       28152     7mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.04-000001        0     r      UNASSIGNED                            
.items-default-000001                                              0     p      STARTED           0    225b 127.0.0.1 xenpher
.items-default-000001                                              0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.22-000011            0     p      STARTED       43184  11.9mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.22-000011            0     r      UNASSIGNED                            
.ds-metrics-elastic_agent.fleet_server-default-2023.05.04-000001   0     p      STARTED      311394  54.2mb 127.0.0.1 xenpher
.ds-metrics-elastic_agent.fleet_server-default-2023.05.04-000001   0     r      UNASSIGNED                            
.ds-logs-system.system-default-2023.05.13-000003                   0     p      STARTED          37 235.8kb 127.0.0.1 xenpher
.ds-logs-system.system-default-2023.05.13-000003                   0     r      UNASSIGNED                            
.fleet-policies-7                                                  0     p      STARTED          47 248.2kb 127.0.0.1 xenpher
.ds-metrics-elastic_agent.metricbeat-default-2023.05.04-000001     0     p      STARTED      880034 236.6mb 127.0.0.1 xenpher
.ds-metrics-elastic_agent.metricbeat-default-2023.05.04-000001     0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.19-000009            0     p      STARTED       60911  12.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.19-000009            0     r      UNASSIGNED                            
.ds-logs-system.system-default-2023.05.14-000004                   0     p      STARTED         393 726.2kb 127.0.0.1 xenpher
.ds-logs-system.system-default-2023.05.14-000004                   0     r      UNASSIGNED                            
.ds-logs-system.syslog-default-2023.05.16-000006                   0     p      STARTED     1075395 190.4mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.16-000006                   0     r      UNASSIGNED                            
.ds-logs-elastic_agent.metricbeat-default-2023.05.12-000003        0     p      STARTED       52626  13.8mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.12-000003        0     r      UNASSIGNED                            
.ds-metrics-windows.service-default-2023.05.19-000001              0     p      STARTED     1188931 147.9mb 127.0.0.1 xenpher
.ds-metrics-windows.service-default-2023.05.19-000001              0     r      UNASSIGNED                            
.fleet-policies-leader-7                                           0     p      STARTED           3   1.9mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.17-000007        0     p      STARTED          49 228.4kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.17-000007        0     r      UNASSIGNED                            
my-data-today-copy-2023.05.12-000001                               0     p      STARTED           0    247b 127.0.0.1 xenpher
my-data-today-copy-2023.05.12-000001                               0     r      UNASSIGNED                            
.fleet-agents-7                                                    0     p      STARTED          14  26.3mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.14-000004                   0     p      STARTED      454633    94mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.14-000004                   0     r      UNASSIGNED                            
.ds-logs-elastic_agent.metricbeat-default-2023.05.17-000008        0     p      STARTED       91480  22.7mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.17-000008        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.13-000003         0     p      STARTED    11030887   4.2gb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.13-000003         0     r      UNASSIGNED                            
.ds-logs-endpoint.events.library-default-2023.05.12-000002         0     p      STARTED       10915   3.4mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.library-default-2023.05.12-000002         0     r      UNASSIGNED                            
.ds-.logs-endpoint.diagnostic.collection-default-2023.05.18-000004 0     p      STARTED          29 737.3kb 127.0.0.1 xenpher
.ds-.logs-endpoint.diagnostic.collection-default-2023.05.18-000004 0     r      UNASSIGNED                            
.ds-logs-elastic_agent.fleet_server-default-2023.05.13-000004      0     p      STARTED         899   1.6mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.13-000004      0     r      UNASSIGNED                            
.geoip_databases                                                   0     p      STARTED          43  46.8mb 127.0.0.1 xenpher
.ds-logs-endpoint.alerts-default-2023.05.17-000003                 0     p      STARTED          20   218kb 127.0.0.1 xenpher
.ds-logs-endpoint.alerts-default-2023.05.17-000003                 0     r      UNASSIGNED                            
.ds-metrics-system.filesystem-default-2023.05.04-000001            0     p      STARTED      352793  83.7mb 127.0.0.1 xenpher
.ds-metrics-system.filesystem-default-2023.05.04-000001            0     r      UNASSIGNED                            
.ds-.logs-deprecation.elasticsearch-default-2023.05.03-000001      0     p      STARTED          12    47kb 127.0.0.1 xenpher
.fleet-actions-7                                                   0     p      STARTED          10  58.1kb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.13-000003                 0     p      STARTED        7799   3.6mb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.13-000003                 0     r      UNASSIGNED                            
.ds-logs-endpoint.events.network-default-2023.05.15-000005         0     p      STARTED       26248   8.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.network-default-2023.05.15-000005         0     r      UNASSIGNED                            
.ds-logs-system.syslog-default-2023.05.04-000001                   0     p      STARTED     4281931 810.8mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.04-000001                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.15-000005        0     p      STARTED         832 721.3kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.15-000005        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.18-000008            0     p      STARTED      159135  34.6mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.18-000008            0     r      UNASSIGNED                            
.ds-logs-endpoint.alerts-default-2023.05.04-000001                 0     p      STARTED         682   1.3mb 127.0.0.1 xenpher
.ds-logs-endpoint.alerts-default-2023.05.04-000001                 0     r      UNASSIGNED                            
.ds-logs-system.system-default-2023.05.04-000001                   0     p      STARTED        9089   3.6mb 127.0.0.1 xenpher
.ds-logs-system.system-default-2023.05.04-000001                   0     r      UNASSIGNED                            
.ds-logs-system.auth-default-2023.05.16-000006                     0     p      STARTED         576 294.1kb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.16-000006                     0     r      UNASSIGNED                            
.ds-logs-system.application-default-2023.05.16-000006              0     p      STARTED          73 213.5kb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.16-000006              0     r      UNASSIGNED                            
.ds-logs-endpoint.events.registry-default-2023.05.16-000006        0     p      STARTED       33562   5.8mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.registry-default-2023.05.16-000006        0     r      UNASSIGNED                            
.ds-logs-system.security-default-2023.05.12-000002                 0     p      STARTED        6968     3mb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.12-000002                 0     r      UNASSIGNED                            
.transform-internal-007                                            0     p      STARTED         134  12.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.14-000004         0     p      STARTED    11068060   4.2gb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.14-000004         0     r      UNASSIGNED
2

this is the rest of the get command

.kibana_8.7.1_001                                                  0     p      UNASSIGNED                            
.ds-logs-elastic_agent.filebeat-default-2023.05.16-000007          0     p      STARTED       33146  10.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.16-000007          0     r      UNASSIGNED                            
.ds-logs-system.system-default-2023.05.16-000006                   0     p      STARTED          48 265.2kb 127.0.0.1 xenpher
.ds-logs-system.system-default-2023.05.16-000006                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.16-000006         0     p      STARTED    10872829   4.1gb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.16-000006         0     r      UNASSIGNED                            
.kibana_task_manager_8.7.1_001                                     0     p      STARTED         754 145.1mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.04-000001          0     p      STARTED       23379   5.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.04-000001          0     r      UNASSIGNED                            
.ds-logs-system.auth-default-2023.05.14-000004                     0     p      STARTED         109 301.7kb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.14-000004                     0     r      UNASSIGNED                            
.ds-logs-system.application-default-2023.05.12-000002              0     p      STARTED          87 196.1kb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.12-000002              0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.22-000012 0     p      STARTED       50828   5.7mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.22-000012 0     r      UNASSIGNED                            
.ds-logs-endpoint.events.network-default-2023.05.12-000002         0     p      STARTED       48092    14mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.network-default-2023.05.12-000002         0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.20-000011 0     p      STARTED        3831 643.1kb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.20-000011 0     r      UNASSIGNED                            
.ds-logs-endpoint.events.network-default-2023.05.13-000003         0     p      STARTED       46526  13.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.network-default-2023.05.13-000003         0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.04-000001 0     p      STARTED       57469   5.9mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.04-000001 0     r      UNASSIGNED                            
.metrics-endpoint.metadata_united_default                          0     p      STARTED          14  12.1mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.api-default-2023.05.04-000001             0     p      STARTED          82 241.4kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.api-default-2023.05.04-000001             0     r      UNASSIGNED                            
.ds-logs-elastic_agent.filebeat-default-2023.05.12-000003          0     p      STARTED       21409   7.1mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.12-000003          0     r      UNASSIGNED                            
.ds-metrics-system.memory-default-2023.05.04-000001                0     p      STARTED      413387  85.1mb 127.0.0.1 xenpher
.ds-metrics-system.memory-default-2023.05.04-000001                0     r      UNASSIGNED                            
.ds-logs-endpoint.events.registry-default-2023.05.12-000002        0     p      STARTED       30861   5.6mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.registry-default-2023.05.12-000002        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.16-000006        0     p      STARTED         359 385.3kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.16-000006        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.15-000005            0     p      STARTED      102063  21.3mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.15-000005            0     r      UNASSIGNED                            
.ds-logs-elastic_agent.metricbeat-default-2023.05.05-000002        0     p      STARTED      352375    62mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.05-000002        0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.18-000008         0     p      STARTED     2028053 853.5mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.18-000008         0     r      UNASSIGNED                            
.fleet-enrollment-api-keys-7                                       0     p      STARTED           3  19.8kb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.15-000006          0     p      STARTED       28123     8mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.15-000006          0     r      UNASSIGNED                            
.ds-logs-system.system-default-2023.05.15-000005                   0     p      STARTED         299 467.2kb 127.0.0.1 xenpher
.ds-logs-system.system-default-2023.05.15-000005                   0     r      UNASSIGNED                            
.ds-logs-elastic_agent.filebeat-default-2023.05.13-000004          0     p      STARTED       21421   6.8mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.13-000004          0     r      UNASSIGNED                            
.ds-logs-endpoint.events.network-default-2023.05.17-000007         0     p      STARTED      965128 225.3mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.network-default-2023.05.17-000007         0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.05-000002                   0     p      STARTED        8114   2.9mb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.05-000002                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.library-default-2023.05.14-000004         0     p      STARTED       10405   3.5mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.library-default-2023.05.14-000004         0     r      UNASSIGNED                            
.ds-.fleet-actions-results-2023.05.04-000001                       0     p      STARTED          10  43.6kb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.05-000002          0     p      STARTED      129270  29.4mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.filebeat-default-2023.05.05-000002          0     r      UNASSIGNED                            
.ds-logs-endpoint.events.library-default-2023.05.16-000006         0     p      STARTED        2387   1.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.library-default-2023.05.16-000006         0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.15-000005         0     p      STARTED    10640012     4gb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.15-000005         0     r      UNASSIGNED                            
.ds-logs-endpoint.events.api-default-2023.05.17-000005             0     p      STARTED           4  57.7kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.api-default-2023.05.17-000005             0     r      UNASSIGNED                            
.ds-logs-system.security-default-2023.05.17-000007                 0     p      STARTED       22244   7.1mb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.17-000007                 0     r      UNASSIGNED                            
.ds-logs-endpoint.events.api-default-2023.05.15-000003             0     p      STARTED          24 111.8kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.api-default-2023.05.15-000003             0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.17-000008                   0     p      STARTED        1081  1022kb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.17-000008                   0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.15-000006 0     p      STARTED       51444   5.5mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.15-000006 0     r      UNASSIGNED                            
.ds-logs-endpoint.events.library-default-2023.05.17-000007         0     p      STARTED      251402  26.2mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.library-default-2023.05.17-000007         0     r      UNASSIGNED                            
.ds-metrics-system.uptime-default-2023.05.04-000001                0     p      STARTED      413443  72.4mb 127.0.0.1 xenpher
.ds-metrics-system.uptime-default-2023.05.04-000001                0     r      UNASSIGNED                            
.ds-logs-elastic_agent.endpoint_security-default-2023.05.17-000008 0     p      STARTED       16328   3.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.17-000008 0     r      UNASSIGNED                            
.ds-logs-system.auth-default-2023.05.17-000007                     0     p      STARTED         510 830.8kb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.17-000007                     0     r      UNASSIGNED                            
.ds-logs-system.security-default-2023.05.14-000004                 0     p      STARTED       10424   4.9mb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.14-000004                 0     r      UNASSIGNED                            
.ds-logs-system.security-default-2023.05.04-000001                 0     p      STARTED      180412  37.3mb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.04-000001                 0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.16-000007                   0     p      STARTED        1086 783.4kb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.16-000007                   0     r      UNASSIGNED                            
.ds-metrics-elastic_agent.filebeat_input-default-2023.05.04-000001 0     p      STARTED      477939 143.9mb 127.0.0.1 xenpher
.ds-metrics-elastic_agent.filebeat_input-default-2023.05.04-000001 0     r      UNASSIGNED                            
.fleet-servers-7                                                   0     p      STARTED           3 973.3kb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.15-000005                   0     p      STARTED      450081    88mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.15-000005                   0     r      UNASSIGNED                            
.ds-metrics-system.fsstat-default-2023.05.04-000001                0     p      STARTED       68942  20.1mb 127.0.0.1 xenpher
.ds-metrics-system.fsstat-default-2023.05.04-000001                0     r      UNASSIGNED                            
.ds-logs-windows.powershell-default-2023.05.19-000001              0     p      STARTED          64 184.3kb 127.0.0.1 xenpher
.ds-logs-windows.powershell-default-2023.05.19-000001              0     r      UNASSIGNED                            
.fleet-artifacts-7                                                 0     p      STARTED          15  14.8kb 127.0.0.1 xenpher
.ds-ilm-history-5-2023.05.03-000001                                0     p      STARTED        1325 340.8kb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.12-000003 0     p      STARTED       48049   9.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.endpoint_security-default-2023.05.12-000003 0     r      UNASSIGNED                            
.ds-metrics-endpoint.policy-default-2023.05.04-000001              0     p      STARTED         107   583kb 127.0.0.1 xenpher
.ds-metrics-endpoint.policy-default-2023.05.04-000001              0     r      UNASSIGNED                            
.ds-logs-elastic_agent-default-2023.05.15-000006                   0     p      STARTED         624 524.6kb 127.0.0.1 xenpher
.ds-logs-elastic_agent-default-2023.05.15-000006                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.file-default-2023.05.16-000006            0     p      STARTED      107763  21.6mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.file-default-2023.05.16-000006            0     r      UNASSIGNED                            
.ds-metrics-windows.perfmon-default-2023.05.19-000001              0     p      STARTED     5092165 333.4mb 127.0.0.1 xenpher
.ds-metrics-windows.perfmon-default-2023.05.19-000001              0     r      UNASSIGNED                            
.ds-logs-endpoint.events.registry-default-2023.05.17-000007        0     p      STARTED     1174751 137.1mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.registry-default-2023.05.17-000007        0     r      UNASSIGNED                            
.apm-agent-configuration                                           0     p      STARTED           0    225b 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.15-000005                 0     p      STARTED        5973   2.6mb 127.0.0.1 xenpher
.ds-logs-system.security-default-2023.05.15-000005                 0     r      UNASSIGNED                            
.ds-metrics-elastic_agent.elastic_agent-default-2023.05.04-000001  0     p      STARTED     1546869   331mb 127.0.0.1 xenpher
.ds-metrics-elastic_agent.elastic_agent-default-2023.05.04-000001  0     r      UNASSIGNED                            
.fleet-files-agent-000001                                          0     p      STARTED           0    225b 127.0.0.1 xenpher
.fleet-file-data-endpoint-000001                                   0     p      STARTED           0    225b 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.16-000007        0     p      STARTED       75218  15.2mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.16-000007        0     r      UNASSIGNED                            
.ds-logs-system.auth-default-2023.05.15-000005                     0     p      STARTED          92 316.5kb 127.0.0.1 xenpher
.ds-logs-system.auth-default-2023.05.15-000005                     0     r      UNASSIGNED                            
.security-profile-8                                                0     p      STARTED           1   8.5kb 127.0.0.1 xenpher
my-data-today-2023.05.12-000001                                    0     p      STARTED           0    225b 127.0.0.1 xenpher
my-data-today-2023.05.12-000001                                    0     r      UNASSIGNED                            
.ds-logs-windows.powershell_operational-default-2023.05.19-000001  0     p      STARTED           1  51.6kb 127.0.0.1 xenpher
.ds-logs-windows.powershell_operational-default-2023.05.19-000001  0     r      UNASSIGNED                            
.ds-logs-system.syslog-default-2023.05.12-000002                   0     p      STARTED      456553  88.6mb 127.0.0.1 xenpher
.ds-logs-system.syslog-default-2023.05.12-000002                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.process-default-2023.05.17-000007         0     p      STARTED     2047512 871.4mb 127.0.0.1 xenpher
.ds-logs-endpoint.events.process-default-2023.05.17-000007         0     r      UNASSIGNED                            
.ds-logs-elastic_agent.fleet_server-default-2023.05.04-000001      0     p      STARTED        1158 510.9kb 127.0.0.1 xenpher
.ds-logs-elastic_agent.fleet_server-default-2023.05.04-000001      0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.22-000010        0     p      STARTED         572 456.2kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.22-000010        0     r      UNASSIGNED                            
.ds-logs-system.application-default-2023.05.14-000004              0     p      STARTED         326 491.2kb 127.0.0.1 xenpher
.ds-logs-system.application-default-2023.05.14-000004              0     r      UNASSIGNED                            
.ds-metrics-endpoint.metadata-default-2023.05.04-000001            0     p      STARTED         123 322.2kb 127.0.0.1 xenpher
.ds-metrics-endpoint.metadata-default-2023.05.04-000001            0     r      UNASSIGNED                            
.ds-logs-system.system-default-2023.05.17-000007                   0     p      STARTED         265 605.6kb 127.0.0.1 xenpher
.ds-logs-system.system-default-2023.05.17-000007                   0     r      UNASSIGNED                            
.ds-logs-endpoint.events.security-default-2023.05.12-000002        0     p      STARTED         459 333.1kb 127.0.0.1 xenpher
.ds-logs-endpoint.events.security-default-2023.05.12-000002        0     r      UNASSIGNED                            
.ds-logs-elastic_agent.metricbeat-default-2023.05.15-000006        0     p      STARTED       63584  14.3mb 127.0.0.1 xenpher
.ds-logs-elastic_agent.metricbeat-default-2023.05.15-000006        0     r      UNASSIGNED
3

What is the full output of the cluster stats API?

4

this is the cluster api

{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "-kMn3M_HS2yUs1AujFSMQg",
  "timestamp" : 1684916360111,
  "status" : "red",
  "indices" : {
    "count" : 205,
    "shards" : {
      "total" : 205,
      "primaries" : 205,
      "replication" : 0.0,
      "index" : {
        "shards" : {
          "min" : 1,
          "max" : 1,
          "avg" : 1.0
        },
        "primaries" : {
          "min" : 1,
          "max" : 1,
          "avg" : 1.0
        },
        "replication" : {
          "min" : 0.0,
          "max" : 0.0,
          "avg" : 0.0
        }
      }
    },
    "docs" : {
      "count" : 144621284,
      "deleted" : 1966
    },
    "store" : {
      "size" : "38.8gb",
      "size_in_bytes" : 41671121224,
      "total_data_set_size" : "38.8gb",
      "total_data_set_size_in_bytes" : 41671121224,
      "reserved" : "0b",
      "reserved_in_bytes" : 0
    },
    "fielddata" : {
      "memory_size" : "992b",
      "memory_size_in_bytes" : 992,
      "evictions" : 0
    },
    "query_cache" : {
      "memory_size" : "0b",
      "memory_size_in_bytes" : 0,
      "total_count" : 0,
      "hit_count" : 0,
      "miss_count" : 0,
      "cache_size" : 0,
      "cache_count" : 0,
      "evictions" : 0
    },
    "completion" : {
      "size" : "0b",
      "size_in_bytes" : 0
    },
    "segments" : {
      "count" : 1576,
      "memory" : "0b",
      "memory_in_bytes" : 0,
      "terms_memory" : "0b",
      "terms_memory_in_bytes" : 0,
      "stored_fields_memory" : "0b",
      "stored_fields_memory_in_bytes" : 0,
      "term_vectors_memory" : "0b",
      "term_vectors_memory_in_bytes" : 0,
      "norms_memory" : "0b",
      "norms_memory_in_bytes" : 0,
      "points_memory" : "0b",
      "points_memory_in_bytes" : 0,
      "doc_values_memory" : "0b",
      "doc_values_memory_in_bytes" : 0,
      "index_writer_memory" : "133.7mb",
      "index_writer_memory_in_bytes" : 140287904,
      "version_map_memory" : "2.1mb",
      "version_map_memory_in_bytes" : 2301097,
      "fixed_bit_set" : "11.2mb",
      "fixed_bit_set_memory_in_bytes" : 11746792,
      "max_unsafe_auto_id_timestamp" : 1684915526604,
      "file_sizes" : { }
    },
    "mappings" : {
      "total_field_count" : 39788,
      "total_deduplicated_field_count" : 17561,
      "total_deduplicated_mapping_size" : "122.7kb",
      "total_deduplicated_mapping_size_in_bytes" : 125705,
      "field_types" : [
        {
          "name" : "alias",
          "count" : 77,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "binary",
          "count" : 4,
          "index_count" : 4,
          "script_count" : 0
        },
        {
          "name" : "boolean",
          "count" : 1460,
          "index_count" : 179,
          "script_count" : 0
        },
        {
          "name" : "byte",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "constant_keyword",
          "count" : 672,
          "index_count" : 174,
          "script_count" : 0
        },
        {
          "name" : "date",
          "count" : 937,
          "index_count" : 183,
          "script_count" : 0
        },
        {
          "name" : "date_nanos",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "date_range",
          "count" : 2,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "double",
          "count" : 207,
          "index_count" : 39,
          "script_count" : 0
        },
        {
          "name" : "double_range",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "flattened",
          "count" : 35,
          "index_count" : 5,
          "script_count" : 0
        },
        {
          "name" : "float",
          "count" : 56,
          "index_count" : 11,
          "script_count" : 0
        },
        {
          "name" : "float_range",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "geo_point",
          "count" : 160,
          "index_count" : 80,
          "script_count" : 0
        },
        {
          "name" : "geo_shape",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "half_float",
          "count" : 3,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "histogram",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "integer",
          "count" : 7,
          "index_count" : 5,
          "script_count" : 0
        },
        {
          "name" : "integer_range",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "ip",
          "count" : 293,
          "index_count" : 175,
          "script_count" : 0
        },
        {
          "name" : "ip_range",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "keyword",
          "count" : 23855,
          "index_count" : 185,
          "script_count" : 0
        },
        {
          "name" : "long",
          "count" : 2321,
          "index_count" : 131,
          "script_count" : 0
        },
        {
          "name" : "long_range",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "match_only_text",
          "count" : 781,
          "index_count" : 115,
          "script_count" : 0
        },
        {
          "name" : "nested",
          "count" : 382,
          "index_count" : 70,
          "script_count" : 0
        },
        {
          "name" : "object",
          "count" : 7164,
          "index_count" : 182,
          "script_count" : 0
        },
        {
          "name" : "scaled_float",
          "count" : 43,
          "index_count" : 6,
          "script_count" : 0
        },
        {
          "name" : "shape",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "short",
          "count" : 4,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "text",
          "count" : 910,
          "index_count" : 177,
          "script_count" : 0
        },
        {
          "name" : "unsigned_long",
          "count" : 209,
          "index_count" : 36,
          "script_count" : 0
        },
        {
          "name" : "version",
          "count" : 2,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "wildcard",
          "count" : 194,
          "index_count" : 48,
          "script_count" : 0
        }
      ],
      "runtime_field_types" : [ ]
    },
    "analysis" : {
      "char_filter_types" : [ ],
      "tokenizer_types" : [ ],
      "filter_types" : [ ],
      "analyzer_types" : [
        {
          "name" : "pattern",
          "count" : 2,
          "index_count" : 2
        }
      ],
      "built_in_char_filters" : [ ],
      "built_in_tokenizers" : [ ],
      "built_in_filters" : [ ],
      "built_in_analyzers" : [
        {
          "name" : "powershell_script_analyzer",
          "count" : 2,
          "index_count" : 2
        }
      ]
    },
    "versions" : [
      {
        "version" : "8.7.1",
        "index_count" : 206,
        "primary_shard_count" : 206,
        "total_primary_size" : "38.8gb",
        "total_primary_bytes" : 41671121224
      }
    ],
    "search" : {
      "total" : 121,
      "queries" : {
        "bool" : 21,
        "terms" : 60,
        "range" : 10,
        "term" : 11
      },
      "sections" : {
        "query" : 71,
        "_source" : 10,
        "pit" : 1,
        "aggs" : 50
      }
    }
  },
  "nodes" : {
    "count" : {
      "total" : 1,
      "coordinating_only" : 0,
      "data" : 1,
      "data_cold" : 1,
      "data_content" : 1,
      "data_frozen" : 1,
      "data_hot" : 1,
      "data_warm" : 1,
      "index" : 0,
      "ingest" : 1,
      "master" : 1,
      "ml" : 1,
      "remote_cluster_client" : 1,
      "search" : 0,
      "transform" : 1,
      "voting_only" : 0
    },
    "versions" : [
      "8.7.1"
    ],
    "os" : {
      "available_processors" : 4,
      "allocated_processors" : 4,
      "names" : [
        {
          "name" : "Linux",
          "count" : 1
        }
      ],
      "pretty_names" : [
        {
          "pretty_name" : "Ubuntu 22.04.2 LTS",
          "count" : 1
        }
      ],
      "architectures" : [
        {
          "arch" : "amd64",
          "count" : 1
        }
      ],
      "mem" : {
        "total" : "11.5gb",
        "total_in_bytes" : 12431908864,
        "adjusted_total" : "11.5gb",
        "adjusted_total_in_bytes" : 12431908864,
        "free" : "657.8mb",
        "free_in_bytes" : 689836032,
        "used" : "10.9gb",
        "used_in_bytes" : 11742072832,
        "free_percent" : 6,
        "used_percent" : 94
      }
    },
    "process" : {
      "cpu" : {
        "percent" : 4
      },
      "open_file_descriptors" : {
        "min" : 1310,
        "max" : 1310,
        "avg" : 1310
      }
    },
    "jvm" : {
      "max_uptime" : "22.5m",
      "max_uptime_in_millis" : 1354084,
      "versions" : [
        {
          "version" : "20.0.1",
          "vm_name" : "OpenJDK 64-Bit Server VM",
          "vm_version" : "20.0.1+9-29",
          "vm_vendor" : "Oracle Corporation",
          "bundled_jdk" : true,
          "using_bundled_jdk" : true,
          "count" : 1
        }
      ],
      "mem" : {
        "heap_used" : "1.9gb",
        "heap_used_in_bytes" : 2077008288,
        "heap_max" : "6gb",
        "heap_max_in_bytes" : 6442450944
      },
      "threads" : 76
    },
    "fs" : {
      "total" : "227.6gb",
      "total_in_bytes" : 244464644096,
      "free" : "58.8gb",
      "free_in_bytes" : 63212478464,
      "available" : "47.2gb",
      "available_in_bytes" : 50719744000
    },
    "plugins" : [ ],
    "network_types" : {
      "transport_types" : {
        "security4" : 1
      },
      "http_types" : {
        "security4" : 1
      }
    },
    "discovery_types" : {
      "single-node" : 1
    },
    "packaging_types" : [
      {
        "flavor" : "default",
        "type" : "deb",
        "count" : 1
      }
    ],
    "ingest" : {
      "number_of_pipelines" : 79,
      "processor_stats" : {
        "append" : {
          "count" : 160,
          "failed" : 0,
          "current" : 0,
          "time" : "3ms",
          "time_in_millis" : 3
        },
        "community_id" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "convert" : {
          "count" : 1243,
          "failed" : 0,
          "current" : 0,
          "time" : "3ms",
          "time_in_millis" : 3
        },
        "date" : {
          "count" : 168202,
          "failed" : 0,
          "current" : 0,
          "time" : "4.2s",
          "time_in_millis" : 4281
        },
        "foreach" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "geoip" : {
          "count" : 10564,
          "failed" : 0,
          "current" : 0,
          "time" : "12s",
          "time_in_millis" : 12030
        },
        "grok" : {
          "count" : 472,
          "failed" : 6,
          "current" : 0,
          "time" : "105ms",
          "time_in_millis" : 105
        },
        "gsub" : {
          "count" : 412,
          "failed" : 0,
          "current" : 0,
          "time" : "1ms",
          "time_in_millis" : 1
        },
        "kv" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "pipeline" : {
          "count" : 168011,
          "failed" : 0,
          "current" : 0,
          "time" : "364ms",
          "time_in_millis" : 364
        },
        "registered_domain" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "remove" : {
          "count" : 336907,
          "failed" : 412,
          "current" : 0,
          "time" : "493ms",
          "time_in_millis" : 493
        },
        "rename" : {
          "count" : 16839,
          "failed" : 5871,
          "current" : 0,
          "time" : "578ms",
          "time_in_millis" : 578
        },
        "script" : {
          "count" : 177787,
          "failed" : 0,
          "current" : 0,
          "time" : "1.1s",
          "time_in_millis" : 1197
        },
        "set" : {
          "count" : 112532,
          "failed" : 0,
          "current" : 0,
          "time" : "1.2s",
          "time_in_millis" : 1215
        },
        "set_security_user" : {
          "count" : 167820,
          "failed" : 0,
          "current" : 0,
          "time" : "1.7s",
          "time_in_millis" : 1703
        },
        "split" : {
          "count" : 5,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        }
      }
    },
    "indexing_pressure" : {
      "memory" : {
        "current" : {
          "combined_coordinating_and_primary" : "0b",
          "combined_coordinating_and_primary_in_bytes" : 0,
          "coordinating" : "0b",
          "coordinating_in_bytes" : 0,
          "primary" : "0b",
          "primary_in_bytes" : 0,
          "replica" : "0b",
          "replica_in_bytes" : 0,
          "all" : "0b",
          "all_in_bytes" : 0
        },
        "total" : {
          "combined_coordinating_and_primary" : "0b",
          "combined_coordinating_and_primary_in_bytes" : 0,
          "coordinating" : "0b",
          "coordinating_in_bytes" : 0,
          "primary" : "0b",
          "primary_in_bytes" : 0,
          "replica" : "0b",
          "replica_in_bytes" : 0,
          "all" : "0b",
          "all_in_bytes" : 0,
          "coordinating_rejections" : 0,
          "primary_rejections" : 0,
          "replica_rejections" : 0
        },
        "limit" : "0b",
        "limit_in_bytes" : 0
      }
    }
  }
}
5

You will need to share the Elasticsearch logs as well.

6

this is the log using tail -200f

	... 191 more
[2023-05-24T15:24:40,720][WARN ][r.suppressed             ] [xenpher] path: /.kibana_8.7.1_001/_pit, params: {index=.kibana_8.7.1_001, keep_alive=10m}
org.elasticsearch.action.search.SearchPhaseExecutionException: 
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:728) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:474) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:217) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:1041) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:769) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$6(TransportSearchAction.java:390) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:112) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:478) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:297) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.TransportOpenPointInTimeAction.doExecute(TransportOpenPointInTimeAction.java:77) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.TransportOpenPointInTimeAction.doExecute(TransportOpenPointInTimeAction.java:37) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:86) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:84) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:163) ~[?:?]
	at org.elasticsearch.action.ActionListener$DelegatingFailureActionListener.onResponse(ActionListener.java:250) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:606) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.SearchRequestCacheDisablingInterceptor.intercept(SearchRequestCacheDisablingInterceptor.java:53) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.ShardSearchRequestInterceptor.intercept(ShardSearchRequestInterceptor.java:24) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.UpdateRequestInterceptor.intercept(UpdateRequestInterceptor.java:27) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.ResizeRequestInterceptor.intercept(ResizeRequestInterceptor.java:98) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.DlsFlsLicenseRequestInterceptor.intercept(DlsFlsLicenseRequestInterceptor.java:106) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.BulkShardRequestInterceptor.intercept(BulkShardRequestInterceptor.java:85) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.IndicesAliasesRequestInterceptor.intercept(IndicesAliasesRequestInterceptor.java:124) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.SearchRequestInterceptor.intercept(SearchRequestInterceptor.java:21) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.runRequestInterceptors(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.handleIndexActionAuthorizationResult(AuthorizationService.java:585) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$11(AuthorizationService.java:486) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:989) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:955) ~[?:?]
	at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$authorizeIndexAction$3(RBACEngine.java:387) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:510) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:111) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:55) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:41) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$CachingAsyncSupplier.getAsync(AuthorizationService.java:1035) ~[?:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.authorizeIndexAction(RBACEngine.java:367) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:479) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:416) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:317) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:150) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$1(CompositeRolesStore.java:201) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$0(RoleReferenceIntersection.java:49) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:55) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromRoleReference(CompositeRolesStore.java:299) ~[?:?]
	at org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$1(RoleReferenceIntersection.java:53) ~[?:?]
	at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
	at org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.buildRole(RoleReferenceIntersection.java:53) ~[?:?]
	at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:216) ~[?:?]
	at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:191) ~[?:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:147) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:319) ~[?:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:159) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:132) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticateAsync(AuthenticatorChain.java:93) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:171) ~[?:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:155) ~[?:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:114) ~[?:?]
	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:84) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:203) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:112) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:90) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:380) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.RestOpenPointInTimeAction.lambda$prepareRequest$0(RestOpenPointInTimeAction.java:44) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:103) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.rest.SecurityRestFilter.doHandleRequest(SecurityRestFilter.java:123) ~[?:?]
	at org.elasticsearch.xpack.security.rest.SecurityRestFilter.lambda$handleRequest$0(SecurityRestFilter.java:113) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.lambda$authenticateAndAttachToContext$2(SecondaryAuthenticator.java:84) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticate(SecondaryAuthenticator.java:94) ~[?:?]
	at org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticateAndAttachToContext(SecondaryAuthenticator.java:78) ~[?:?]
	at org.elasticsearch.xpack.security.rest.SecurityRestFilter.lambda$handleRequest$2(SecurityRestFilter.java:107) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:132) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onResponse(ActionListener.java:440) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.writeAuthToContext(AuthenticatorChain.java:346) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.finishAuthentication(AuthenticatorChain.java:325) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.maybeLookupRunAsUser(AuthenticatorChain.java:202) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$doAuthenticate$1(AuthenticatorChain.java:121) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:141) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$getAuthenticatorConsumer$4(AuthenticatorChain.java:188) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.ServiceAccountAuthenticator.lambda$authenticate$0(ServiceAccountAuthenticator.java:55) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.service.ServiceAccountService.lambda$authenticateToken$0(ServiceAccountService.java:136) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:141) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CachingServiceAccountTokenStore.lambda$authenticateWithCache$1(CachingServiceAccountTokenStore.java:100) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:510) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:111) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:55) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.service.CachingServiceAccountTokenStore.authenticateWithCache(CachingServiceAccountTokenStore.java:98) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CachingServiceAccountTokenStore.authenticate(CachingServiceAccountTokenStore.java:82) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CompositeServiceAccountTokenStore.lambda$authenticate$0(CompositeServiceAccountTokenStore.java:37) ~[?:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:135) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CachingServiceAccountTokenStore.lambda$authenticateWithCache$2(CachingServiceAccountTokenStore.java:118) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.service.FileServiceAccountTokenStore.doAuthenticate(FileServiceAccountTokenStore.java:82) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CachingServiceAccountTokenStore.authenticateWithCache(CachingServiceAccountTokenStore.java:110) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CachingServiceAccountTokenStore.authenticate(CachingServiceAccountTokenStore.java:82) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CompositeServiceAccountTokenStore.lambda$authenticate$0(CompositeServiceAccountTokenStore.java:37) ~[?:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:117) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.CompositeServiceAccountTokenStore.authenticate(CompositeServiceAccountTokenStore.java:44) ~[?:?]
	at org.elasticsearch.xpack.security.authc.service.ServiceAccountService.authenticateToken(ServiceAccountService.java:134) ~[?:?]
	at org.elasticsearch.xpack.security.authc.ServiceAccountAuthenticator.authenticate(ServiceAccountAuthenticator.java:53) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$getAuthenticatorConsumer$5(AuthenticatorChain.java:180) ~[?:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:117) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.doAuthenticate(AuthenticatorChain.java:136) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticateAsync(AuthenticatorChain.java:95) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:149) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:127) ~[?:?]
	at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:101) ~[?:?]
	at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:402) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:532) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:317) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:379) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:460) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:353) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.handlePipelinedRequest(Netty4HttpPipeliningHandler.java:128) ~[?:?]
	at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:118) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1373) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:1623) ~[?:?]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.kibana_8.7.1_001][0]]. Consider using `allow_partial_search_results` setting to bypass this error.
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:245) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:469) ~[elasticsearch-8.7.1.jar:?]
	... 191 more
7

We need to see the logs from startup to see what caused the cluster to be red.

8

what command should i use because cat show the same output

9

Find the logs on the file system and post them here. The location will depend on how you installed Elasticsearch.

10

i copy the log file to my github because the file is too big and when i try to send it here it say server error.
github link

Screenshot from 2023-05-24 16-08-58
Screenshot from 2023-05-24 16-08-581016×678 95.6 KB

11

As far as I can see that is only a set of large stack traces from failed searches. There is nothing that indicates what the problem is as far as I can see. Can you restart the node and post the logs from the point where Elasticsearch restarts?

12

this is the log after stopping and starting Elasticsearch

[2023-05-24T16:23:37,071][INFO ][o.e.n.Node               ] [xenpher] stopped
[2023-05-24T16:23:37,072][INFO ][o.e.n.Node               ] [xenpher] closing ...
[2023-05-24T16:23:37,328][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [xenpher] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-9424520871293881299/geoip-databases/Jn2KxqKoTIeiVfCrZZApgw/GeoLite2-Country.mmdb]
[2023-05-24T16:23:37,329][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [xenpher] evicted [227] entries from cache after reloading database [/tmp/elasticsearch-9424520871293881299/geoip-databases/Jn2KxqKoTIeiVfCrZZApgw/GeoLite2-ASN.mmdb]
[2023-05-24T16:23:37,330][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [xenpher] evicted [227] entries from cache after reloading database [/tmp/elasticsearch-9424520871293881299/geoip-databases/Jn2KxqKoTIeiVfCrZZApgw/GeoLite2-City.mmdb]
[2023-05-24T16:23:37,335][INFO ][o.e.n.Node               ] [xenpher] closed
[2023-05-24T16:24:15,110][INFO ][o.e.n.Node               ] [xenpher] version[8.7.1], pid[997039], build[deb/f229ed3f893a515d590d0f39b05f68913e2d9b53/2023-04-27T04:33:42.127815583Z], OS[Linux/5.19.0-41-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/20.0.1/20.0.1+9-29]
[2023-05-24T16:24:15,142][INFO ][o.e.n.Node               ] [xenpher] JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]
[2023-05-24T16:24:15,142][INFO ][o.e.n.Node               ] [xenpher] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -Xms6g, -Xmx6g, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-4127784203575546257, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=3221225472, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=deb, --module-path=/usr/share/elasticsearch/lib, --add-modules=jdk.net, -Djdk.module.main=org.elasticsearch.server]
[2023-05-24T16:24:26,428][INFO ][c.a.c.i.j.JacksonVersion ] [xenpher] Package versions: jackson-core=2.13.4, jackson-databind=2.13.4-2, jackson-dataformat-xml=2.13.4, jackson-datatype-jsr310=2.13.4, azure-core=1.34.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot
[2023-05-24T16:24:33,944][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [aggregations]
[2023-05-24T16:24:33,945][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [analysis-common]
[2023-05-24T16:24:33,945][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [apm]
[2023-05-24T16:24:33,945][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [blob-cache]
[2023-05-24T16:24:33,946][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [constant-keyword]
[2023-05-24T16:24:33,946][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [data-streams]
[2023-05-24T16:24:33,946][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [frozen-indices]
[2023-05-24T16:24:33,946][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [ingest-attachment]
[2023-05-24T16:24:33,947][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [ingest-common]
[2023-05-24T16:24:33,947][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [ingest-geoip]
[2023-05-24T16:24:33,947][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [ingest-user-agent]
[2023-05-24T16:24:33,947][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [kibana]
[2023-05-24T16:24:33,948][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [lang-expression]
[2023-05-24T16:24:33,948][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [lang-mustache]
[2023-05-24T16:24:33,948][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [lang-painless]
[2023-05-24T16:24:33,949][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [legacy-geo]
[2023-05-24T16:24:33,949][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [mapper-extras]
[2023-05-24T16:24:33,949][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [mapper-version]
[2023-05-24T16:24:33,949][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [old-lucene-versions]
[2023-05-24T16:24:33,950][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [parent-join]
[2023-05-24T16:24:33,950][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [percolator]
[2023-05-24T16:24:33,950][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [rank-eval]
[2023-05-24T16:24:33,951][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [reindex]
[2023-05-24T16:24:33,951][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [repositories-metering-api]
[2023-05-24T16:24:33,951][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [repository-azure]
[2023-05-24T16:24:33,951][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [repository-gcs]
[2023-05-24T16:24:33,952][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [repository-s3]
[2023-05-24T16:24:33,952][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [repository-url]
[2023-05-24T16:24:33,952][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [runtime-fields-common]
[2023-05-24T16:24:33,953][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [search-business-rules]
[2023-05-24T16:24:33,953][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [searchable-snapshots]
[2023-05-24T16:24:33,953][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [snapshot-based-recoveries]
[2023-05-24T16:24:33,953][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [snapshot-repo-test-kit]
[2023-05-24T16:24:33,954][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [spatial]
[2023-05-24T16:24:33,954][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [systemd]
[2023-05-24T16:24:33,954][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [transform]
[2023-05-24T16:24:33,955][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [transport-netty4]
[2023-05-24T16:24:33,955][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [unsigned-long]
[2023-05-24T16:24:33,955][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [vector-tile]
[2023-05-24T16:24:33,955][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [wildcard]
[2023-05-24T16:24:33,955][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-aggregate-metric]
[2023-05-24T16:24:33,956][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-analytics]
[2023-05-24T16:24:33,956][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-async]
[2023-05-24T16:24:33,956][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-async-search]
[2023-05-24T16:24:33,956][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-autoscaling]
[2023-05-24T16:24:33,957][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-ccr]
[2023-05-24T16:24:33,957][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-core]
[2023-05-24T16:24:33,957][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-deprecation]
[2023-05-24T16:24:33,957][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-enrich]
[2023-05-24T16:24:33,957][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-eql]
[2023-05-24T16:24:33,958][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-fleet]
[2023-05-24T16:24:33,958][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-graph]
[2023-05-24T16:24:33,958][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-identity-provider]
[2023-05-24T16:24:33,958][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-ilm]
[2023-05-24T16:24:33,959][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-logstash]
[2023-05-24T16:24:33,959][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-ml]
[2023-05-24T16:24:33,959][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-monitoring]
[2023-05-24T16:24:33,959][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-profiling]
[2023-05-24T16:24:33,960][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-ql]
[2023-05-24T16:24:33,960][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-rollup]
[2023-05-24T16:24:33,960][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-security]
[2023-05-24T16:24:33,961][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-shutdown]
[2023-05-24T16:24:33,961][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-sql]
[2023-05-24T16:24:33,961][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-stack]
[2023-05-24T16:24:33,961][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-text-structure]
[2023-05-24T16:24:33,962][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-voting-only-node]
[2023-05-24T16:24:33,962][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-watcher]
[2023-05-24T16:24:33,962][INFO ][o.e.p.PluginsService     ] [xenpher] loaded module [x-pack-write-load-forecaster]
[2023-05-24T16:24:33,963][INFO ][o.e.p.PluginsService     ] [xenpher] no plugins loaded
[2023-05-24T16:24:42,145][INFO ][o.e.e.NodeEnvironment    ] [xenpher] using [1] data paths, mounts [[/ (/dev/sda3)]], net usable_space [47.5gb], net total_space [227.6gb], types [ext4]
[2023-05-24T16:24:42,146][INFO ][o.e.e.NodeEnvironment    ] [xenpher] heap size [6gb], compressed ordinary object pointers [true]
[2023-05-24T16:24:47,213][INFO ][o.e.n.Node               ] [xenpher] node name [xenpher], node ID [Jn2KxqKoTIeiVfCrZZApgw], cluster name [elasticsearch], roles [data_cold, data, remote_cluster_client, master, data_warm, data_content, transform, data_hot, ml, data_frozen, ingest]
[2023-05-24T16:24:53,209][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [xenpher] [controller/997061] [Main.cc@123] controller (64 bit): Version 8.7.1 (Build e429aa288c3883) Copyright (c) 2023 Elasticsearch BV
[2023-05-24T16:24:55,519][INFO ][o.e.x.s.Security         ] [xenpher] Security is enabled
[2023-05-24T16:24:56,600][INFO ][o.e.x.s.a.s.FileRolesStore] [xenpher] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2023-05-24T16:24:57,259][INFO ][o.e.x.s.InitialNodeSecurityAutoConfiguration] [xenpher] Auto-configuration will not generate a password for the elastic built-in superuser, as we cannot  determine if there is a terminal attached to the elasticsearch process. You can use the `bin/elasticsearch-reset-password` tool to set the password for the elastic user.
[2023-05-24T16:24:57,926][INFO ][o.e.x.p.ProfilingPlugin  ] [xenpher] Profiling is enabled
[2023-05-24T16:25:00,123][INFO ][o.e.t.n.NettyAllocator   ] [xenpher] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2023-05-24T16:25:00,204][INFO ][o.e.i.r.RecoverySettings ] [xenpher] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2023-05-24T16:25:00,305][INFO ][o.e.d.DiscoveryModule    ] [xenpher] using discovery type [single-node] and seed hosts providers [settings]
[2023-05-24T16:25:02,626][INFO ][o.e.n.Node               ] [xenpher] initialized
[2023-05-24T16:25:02,627][INFO ][o.e.n.Node               ] [xenpher] starting ...
[2023-05-24T16:25:02,802][INFO ][o.e.x.s.c.f.PersistentCache] [xenpher] persistent cache index loaded
[2023-05-24T16:25:02,804][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [xenpher] deprecation component started
[2023-05-24T16:25:03,219][INFO ][o.e.t.TransportService   ] [xenpher] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2023-05-24T16:25:07,891][WARN ][o.e.c.c.ClusterBootstrapService] [xenpher] this node is locked into cluster UUID [-kMn3M_HS2yUs1AujFSMQg] but [cluster.initial_master_nodes] is set to [xenpher]; remove this setting to avoid possible data loss caused by subsequent cluster bootstrap attempts; for further information see https://www.elastic.co/guide/en/elasticsearch/reference/8.7/important-settings.html#initial_master_nodes
[2023-05-24T16:25:08,157][INFO ][o.e.c.s.MasterService    ] [xenpher] elected-as-master ([1] nodes joined)[_FINISH_ELECTION_, {xenpher}{Jn2KxqKoTIeiVfCrZZApgw}{FU2bgVqZSKCSdHZWg1KxIQ}{xenpher}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.7.1} completing election], term: 16, version: 2923, delta: master node changed {previous [], current [{xenpher}{Jn2KxqKoTIeiVfCrZZApgw}{FU2bgVqZSKCSdHZWg1KxIQ}{xenpher}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.7.1}]}
[2023-05-24T16:25:08,623][INFO ][o.e.c.s.ClusterApplierService] [xenpher] master node changed {previous [], current [{xenpher}{Jn2KxqKoTIeiVfCrZZApgw}{FU2bgVqZSKCSdHZWg1KxIQ}{xenpher}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.7.1}]}, term: 16, version: 2923, reason: Publication{term=16, version=2923}
[2023-05-24T16:25:08,750][INFO ][o.e.r.s.FileSettingsService] [xenpher] starting file settings watcher ...
[2023-05-24T16:25:08,774][INFO ][o.e.r.s.FileSettingsService] [xenpher] file settings service up and running [tid=54]
[2023-05-24T16:25:08,781][INFO ][o.e.h.AbstractHttpServerTransport] [xenpher] publish_address {192.168.88.240:9200}, bound_addresses {[::]:9200}
[2023-05-24T16:25:08,782][INFO ][o.e.n.Node               ] [xenpher] started {xenpher}{Jn2KxqKoTIeiVfCrZZApgw}{FU2bgVqZSKCSdHZWg1KxIQ}{xenpher}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.7.1}{xpack.installed=true, ml.allocated_processors_double=4.0, ml.max_jvm_size=6442450944, ml.allocated_processors=4, ml.machine_memory=12431908864}
[2023-05-24T16:25:08,795][INFO ][o.e.c.c.NodeJoinExecutor ] [xenpher] node-join: [{xenpher}{Jn2KxqKoTIeiVfCrZZApgw}{FU2bgVqZSKCSdHZWg1KxIQ}{xenpher}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.7.1}] with reason [completing election]
[2023-05-24T16:25:11,492][WARN ][o.e.x.s.a.ApiKeyAuthenticator] [xenpher] Authentication using apikey failed - apikey authentication for id Y_uzKIgBgbYXmJZiIrq5 encountered a failure
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
	at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:177) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.checkGlobalBlock(TransportSingleShardAction.java:111) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.<init>(TransportSingleShardAction.java:145) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:97) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:50) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:86) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:84) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:163) ~[?:?]
	at org.elasticsearch.action.ActionListener$DelegatingFailureActionListener.onResponse(ActionListener.java:250) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:606) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.SearchRequestCacheDisablingInterceptor.intercept(SearchRequestCacheDisablingInterceptor.java:53) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.ShardSearchRequestInterceptor.intercept(ShardSearchRequestInterceptor.java:24) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.UpdateRequestInterceptor.intercept(UpdateRequestInterceptor.java:27) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.ResizeRequestInterceptor.intercept(ResizeRequestInterceptor.java:98) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.DlsFlsLicenseRequestInterceptor.intercept(DlsFlsLicenseRequestInterceptor.java:106) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.BulkShardRequestInterceptor.intercept(BulkShardRequestInterceptor.java:85) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.IndicesAliasesRequestInterceptor.intercept(IndicesAliasesRequestInterceptor.java:124) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:604) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) ~[?:?]
	at org.elasticsearch.xpack.security.authz.interceptor.SearchRequestInterceptor.intercept(SearchRequestInterceptor.java:21) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.runRequestInterceptors(AuthorizationService.java:600) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.handleIndexActionAuthorizationResult(AuthorizationService.java:585) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$11(AuthorizationService.java:486) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:989) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:955) ~[?:?]
	at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$authorizeIndexAction$3(RBACEngine.java:387) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:510) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:111) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:55) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:41) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService$CachingAsyncSupplier.getAsync(AuthorizationService.java:1035) ~[?:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.authorizeIndexAction(RBACEngine.java:367) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:479) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:416) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:317) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:150) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$1(CompositeRolesStore.java:201) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:209) ~[?:?]
	at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:191) ~[?:?]
	at org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:147) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:319) ~[?:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:159) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:167) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:132) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticateAsync(AuthenticatorChain.java:93) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:171) ~[?:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:155) ~[?:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$1(SecurityActionFilter.java:110) ~[?:?]
	at org.elasticsearch.xpack.core.security.SecurityContext.executeAsInternalUser(SecurityContext.java:165) ~[?:?]
	at org.elasticsearch.xpack.security.authz.AuthorizationUtils.switchUserBasedOnActionOriginAndExecute(AuthorizationUtils.java:121) ~[?:?]
	at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:106) ~[?:?]
	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:84) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:203) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:112) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:90) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:380) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.client.internal.support.AbstractClient.get(AbstractClient.java:480) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:216) ~[?:?]
	at org.elasticsearch.xpack.security.authc.ApiKeyService.loadApiKeyAndValidateCredentials(ApiKeyService.java:702) ~[?:?]
	at org.elasticsearch.xpack.security.authc.ApiKeyService.tryAuthenticate(ApiKeyService.java:658) ~[?:?]
	at org.elasticsearch.xpack.security.authc.ApiKeyAuthenticator.authenticate(ApiKeyAuthenticator.java:51) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$getAuthenticatorConsumer$5(AuthenticatorChain.java:180) ~[?:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:135) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$getAuthenticatorConsumer$5(AuthenticatorChain.java:158) ~[?:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:135) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$getAuthenticatorConsumer$5(AuthenticatorChain.java:158) ~[?:?]
	at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:117) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.doAuthenticate(AuthenticatorChain.java:136) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticateAsync(AuthenticatorChain.java:95) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:149) ~[?:?]
	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:127) ~[?:?]
	at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:101) ~[?:?]
	at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:402) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:532) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:317) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:379) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:460) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:353) ~[elasticsearch-8.7.1.jar:?]
	at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.handlePipelinedRequest(Netty4HttpPipeliningHandler.java:128) ~[?:?]
	at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:118) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1373) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:1623) ~[?:?]

i found out after that log the rest showing Authentication using apikey failed for the same id

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.