New Unassigned shards every day

Elastic Stack Elasticsearch
1

Hi

I have installed ELK stack and it has been running great.
I followed this guide for the setup: http://pfelk.3ilson.com/

I needed a grok debugger and therefore I installed x-pack with a basic license.
When installing I disabled security and then I named the cluster and node, even though I only have one node right now.

when looking in the kibana monitoring I see "unassigned shards" for each indice.

It seems like it is for all newly created shards.

Through a bit of Googling I manage to get the shards assigned running this command:
curl -XPUT 'localhost:9200/_settings' -d '
{
"index" : {
"number_of_replicas" : 0
}
}'

But everyday I have to do it, how can I make the change permanent?
Does any one know why it is happening?

2

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats and APM feel left out! :wink:

By default the Logstash template includes one replica. You will need to edit the template and change that so that future indices only include the primary. Note though that having replicas unassigned when you are running a single node doesn't cause any problems :slight_smile:

3

I am as you might guess new to Elastic Stack and I actually don't think I am using a template.
If not is it then using one of those in this directory:
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.4.2-java/lib/logstash/outputs/elasticsearch
(Ubuntu install)

The file elasticsearch-template-es5x.json looks like this:
{
"template" : "logstash-",
"version" : 50001,
"settings" : {
"index.refresh_interval" : "5s"
},
"mappings" : {
"default" : {
"_all" : {"enabled" : true, "norms" : false},
"dynamic_templates" : [ {
"message_field" : {
"path_match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text",
"norms" : false
}
}
}, {
"string_fields" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text", "norms" : false,
"fields" : {
"keyword" : { "type": "keyword", "ignore_above": 256 }
}
}
}
} ],
"properties" : {
"@timestamp": { "type": "date", "include_in_all": false },
"@version": { "type": "keyword", "include_in_all": false },
"geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "half_float" },
"longitude" : { "type" : "half_float" }
}
}
}
}
}

I can't see anything referring to shards, so I don't know what to change?

4

What does GET _templates/ show? You may not want to post the whole thing, just look for the Logstash one :slight_smile:

5

I am getting this in return.

{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "No endpoint or operation is available at [_templates]"
}
],
"type": "illegal_argument_exception",
"reason": "No endpoint or operation is available at [_templates]"
},
"status": 400
}

6

Sorry, it's just _template :slight_smile:

7

Sorry for got the last /

"logstash": {
"order": 0,
"version": 50001,
"template": "logstash-",
"settings": {
"index": {
"refresh_interval": "5s"
}
},
"mappings": {
"default": {
"_all": {
"enabled": true,
"norms": false
},
"dynamic_templates": [
{
"message_field": {
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"type": "text",
"norms": false
}
}
},
{
"string_fields": {
"match": "",
"match_mapping_type": "string",
"mapping": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
],
"properties": {
"@timestamp": {
"type": "date",
"include_in_all": false
},
"@version": {
"type": "keyword",
"include_in_all": false
},
"geoip": {
"dynamic": true,
"properties": {
"ip": {
"type": "ip"
},
"location": {
"type": "geo_point"
},
"latitude": {
"type": "half_float"
},
"longitude": {
"type": "half_float"
}
}
}
}
}
},
"aliases": {}
},
".ml-meta": {
"order": 0,
"version": 5060499,
"template": ".ml-meta",
"settings": {
"index": {
"mapper": {
"dynamic": "true"
},
"mapping": {
"single_type": "true"
},
"unassigned": {
"node_left": {
"delayed_timeout": "1m"
}
},
"number_of_shards": "1"
}
},
"mappings": {
"doc": {
"dynamic_templates": [
{
"strings_as_keywords": {
"match": "*",
"mapping": {
"type": "keyword"
}
}
}
]
}
},
"aliases": {}
},

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.