Kibana slow response via kibana console

[Sam_Rybkin]

Hi we use docker-compose for our stack. And our kibana console start to work very slow.
Kibana version 7.9.2.
curl -k cluster/health?pretty

{
  "cluster_name" : "some-dev-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 1,       
  "active_primary_shards" : 359,
  "active_shards" : 359,
  "relocating_shards" : 0,           
  "initializing_shards" : 0,
  "unassigned_shards" : 38,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 90.42821158690177
}

#iostat
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           5.06    0.00    0.52    2.25    0.00   92.17

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sda             205.24       223.78      1417.58 22385804661 141808151591
scd0              0.00         0.00         0.00       1028          0
dm-0            206.39       223.69      1417.49 22376397893 141799227892
dm-1              0.05         0.09         0.09    9391360    8913308

Can you advise how we can fix this issue ? If needed to add some information can advise what exactly need to send ?

Will be thankful for any information and any advice.

[carly.richmond]

Hi @Sam_Rybkin,

Welcome to the community! I see from your cluster health output that the status is yellow:

@Sam_Rybkin

"status" : "yellow",

It could be the case that this is causing your issues. I would recommend checking the state of your shards and check if any of the proposed fixes resolve your issue as per the troubleshooting guide.

Just an aside that the version of Kibana you're using is past EOL, so we would recommend upgrading when you can.

Hope that helps!

[Sam_Rybkin]

@carly.richmond

It could be the case that this is causing your issues. I would recommend checking the state of your shards and check if any of the proposed fixes resolve your issue as per the troubleshooting guide.

Just an aside that the version of Kibana you're using is past EOL, so we would recommend upgrading when you can.

Hope that helps!

Hi @carly.richmond !! Thanks for reply !!!
I am checking the cluster health status:

GET _cluster/health?filter_path=status,*_shards
{
  "status" : "yellow",
  "active_primary_shards" : 362,
  "active_shards" : 362,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 40,
  "delayed_unassigned_shards" : 0
}

And I see that we have "unassigned_shards" : 40, could be it reason slow working kibana console?

I am checking index:

GET _cluster/allocation/explain?filter_path=index,node_allocation_decisions.node_name,node_allocation_decisions.deciders.*
{
  "index": "test-2024.06.11",
  "shard": 0,
  "primary": false
}

And I receive next explanation:
"explanation" : "a copy of this shard is already allocated to this node [[test-2024.06.11][0]

How we can fix this issue ?

Also we have only one data node ("number_of_data_nodes" : 1,)

[Sam_Rybkin]

Hi @carly.richmond !

We changed our settings in kibana:

PUT /.kibana/_settings
{
 "index" : {
  "number_of_replicas":0,              
  "auto_expand_replicas": false
 }
}

Because we have only one data_node. ( "number_of_data_nodes" : 1,)

But our Kibana console working very slow. Maybe you have any advices and recommendation how we can speed up our kibana console ?

Also what we can do with "unassigned_shards" ?
If they have "explanation" : "a copy of this shard is already allocated to this node ? After changes what we did.

[Sam_Rybkin]

Can someone help us ? We still receiving issue with slow working kibana console

[carly.richmond]

Hi @Sam_Rybkin,

Can you confirm if your cluster is now in a green state? Or is it still green?

What queries/ operations are you running via the DevTools console? Can you please share?