Elasticsearch data nodes total size

Elastic Stack Elasticsearch
1

How to get total/average data nodes size through API's?

The below api to get the data nodes allocation each data nodes but i need average or total size of data nodes. Please help me how to get this.

http://localhost:9200/_cat/allocation
shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
29 397.4gb 556.6gb 1.2tb 1.8tb 29 10.236.140.43 10.236.140.43 vc2coma1081153n-node-02
30 469.5gb 1014.2gb 863gb 1.8tb 54 10.240.138.112 10.240.138.112 vc2coma2081152n-node-03
29 445.2gb 545.7gb 1.3tb 1.8tb 29 10.240.148.27 10.240.148.27 vc2coma2085406n-node-05
29 393.9gb 741.4gb 197gb 938.5gb 79 10.240.146.21 10.240.146.21 vc2coma2080685n-node-01
29 395gb 685.2gb 253.3gb 938.5gb 73 10.240.146.45 10.240.146.45 vc2coma2080687n-node-04

2

try the API _stats

http://localhost:9200/_stats?all=true

look for store size in bytes near the top. (and divide by 1024000000 for Gig. )
Here is my return

{
"_shards": {
"total": 108,
"successful": 108,
"failed": 0
},
"_all": {
"primaries": {
"docs": {
"count": 560336831,
"deleted": 17200
},
"store": {
"size_in_bytes": 257292738493,
"throttle_time_in_millis": 0
},

3

Thanks for your reply looks like my store bytes is not matching. Below are my details.

Allocation details :-
shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
31 485.2gb 775.5gb 163gb 938.5gb 82 10.240.146.45 10.240.146.45 vc2coma2080687n-node-04
32 491.7gb 839.5gb 99gb 938.5gb 89 10.240.146.21 10.240.146.21 vc2coma2080685n-node-01
32 569.3gb 670.2gb 1.1tb 1.8tb 35 10.240.148.27 10.240.148.27 vc2coma2085406n-node-05
31 499.1gb 658.3gb 1.1tb 1.8tb 35 10.236.140.43 10.236.140.43 vc2coma1081153n-node-02
32 591.6gb 1.1tb 740.9gb 1.8tb 60 10.240.138.112 10.240.138.112 vc2coma2081152n-node-03

My store info :-

"_shards": {
"total": 158,
"successful": 158,
"failed": 0
},
"_all": {
"primaries": {
"docs": {
"count": 1054938644,
"deleted": 0
},
"store": {
"size_in_bytes": 1410686115558,
"throttle_time_in_millis": 0
}

I calculated disk used column in allocation info. It's coming more than 4 tb but in store showing 1.4 tb. which one is the correct store data.

4

have you loaded the HQ plugin?
HQ will quickly tell you the size to compare with what the API returns.

For instance one of my clusters shows in HQ

The API returns

1277160888 / 1024000000=1.24G

5

Thanks Mark for HQ plugin tool. I have installed KOPF plugin in elasticsearch.

Just now i have installed HQ as well. the below are document and size of data.

Store value is not matching with above figures.

6

Not sure but I think HQ size is only the primaries and not the replicas.
I know in Kopf that the replicas are taken into account. (as seen in your screen cap above. )

Are any indices closed? Closed indices are stored on disk and take up more space but may be included in KPOF stats. I'm not sure?

Anyone else know the difference?