How many records a shard can have?

Elastic Stack Elasticsearch
1

What space or records a shard can have?

How to reallocate shards when primary shard is full?

How much data a index can have?

Thanks in advance

2

A shard can hold a max of 2 billions docs. Other than that there's no size limit, but we recommend a max of less than 50GB as larger shards can mean slower (re)allocation.

3

How to perform shards re-allocation if i have 90gb data and I am getting primary shard inactive exception

4

Please post the entire error you are seeing.

5

[2017-10-24T12:29:47,946][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 503 (
{"type"=>"unavailable_shards_exception", "reason"=>"[jdbc_metadata_pt_new_logs][2] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[jdbc_metadata_pt_new_logs][2]] containing [27] requests]"}
)

6

What do the Elasticsearch logs show?

7

I am unable to see that right now

8

What does _cat/allocation show?

9

you can re-allocate the shard settings as per your disk size allocation.Please refer below link:
https://www.elastic.co/guide/en/elasticsearch/reference/5.5/disk-allocator.html

11

I have already set _cluster/settings
still I am getting same error

12

_cat/allocation is like this

823 5.9gb 74.1gb 24.3gb 98.4gb 75 ipaddress ipaddress 4kL3f1b
829 UNASSIGNED

13

Do you have a single node?

14

Yes.

15

please share complete configuration of your set up and logs details for more understanding.

16

Ok, well you probably have too many shards. It may not be causing this problem but it doesn't help.

If you can't see the Elasticsearch logs it makes things hard. What if you did a https://www.elastic.co/guide/en/elasticsearch/reference/5.6/cluster-allocation-explain.html on the index in question, what does that output?

17

I guess m getting this problem because my machine is out of disk space

18

seeing your previous error message it seems there is more shards but you have configured only one node due to this shards have no enough space for allocation.
for resolving this i prefer you to configure more nodes for the same.

19

Yes, that would make sense.

20

Do i need to enable more nodes to allocate shards other than primary one

21

Yes, for resolving allocation issue you have to enable one more node at least i.e good to configure elasticsearch cluster for distributing shards among them removing disk space.
also you can refer below link: