ElasticSearch on Yellow status && "unassigned_shards" : 985


(Efrain Mesia) #1

Hey guys,

I configured an ELK stack server (CentOS), I configured winlogbeat in one windows with the default parameter. After I started streaming the logs to the ELK server, the server got stuck. I guess the reason its too much logs sent to the ELK server.
I runned the command:
curl -XGET 'localhost:9200/_cluster/health?pretty'
the Output:
"cluster_name" : "elasticsearch", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 1007, "active_shards" : 1007, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 985, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 50.55220883534136

There is a lot of unassigned_shards, Is it possible that make the problem?

Do anyone have a dummy configuration to set the .yml in my windows server?


(Ignacio Vera) #2

I think the problem is not the amount of logs but the amount of replicas assigned when the index are created. Because you have only one data node, you cannot have replicas because primary and replicas cannot be in the same node. Therefore you have all those unassigned shards.

I believe you need to change the template settings in winlogbeat so index are created without replicas:

https://www.elastic.co/guide/en/beats/winlogbeat/current/configuration-template.html


(Efrain Mesia) #3

Thank you, Ignacio.

The changes I need to do is in the ELK stack?


(Ignacio Vera) #4

I think they need to be done on the winlogbeat. See the link above.


(Ignacio Vera) #5

Of course, if you want to change the setting in existing indices you need to do it in elasticsearch using the following API:

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html


(Efrain Mesia) #6

It didnt help me at all, when i start the service Elastic search i get logs.

what can i do?


(Ignacio Vera) #7

Sorry to hear that, could you provide more information about what you have done so we can get a better picture?

It would be good if you share how your index looks like, using the indices API. For example"

GET <your-index>

(Efrain Mesia) #8

Im sorry for being annoying.
How am i suppose to do that?
From the LinuxServer or from kibana?


(Ignacio Vera) #9

From Kibana, if you want to do it from a LinuxServer you would need to do it using CURL:

curl -X GET "your-host:your-port/your-index"

for example:

curl -X GET "localhost:9200/apm-telemetry"


(Efrain Mesia) #10

its a really long output.
Anyways, Im still testing this ELK server. Maybe the best way to delete everything...


(Ignacio Vera) #11

If you are testing I would try to start again making sure that the following setting in winlogbeat.yml is set to 0:

index.number_of_replicas: 0

If you only have a node, it does not make sense to have replicas.


(Efrain Mesia) #12

there is a way to clear all indexes and make it from scratch?


(Ignacio Vera) #13

You can do it using a wildcard:

curl -X DELETE "localhost:9200/*"

Note that this will delete the Kibana index so you will need to restart it again so the index is recreated.


(Efrain Mesia) #14

That will remove all the indexes and all the shards?
Do you think my elk will go back to life?


(Ignacio Vera) #15

Yes, that should remove all data from the cluster.


(Efrain Mesia) #16

Thanks for the help.