Shared limit

i got this error when i tried to add some indices in AWS-ES
but this cluster currently has [1992]/[2000] maximum shards open
how can i solve this, if i want to keep that ES creates a delay index for each logs

Increase the number of nodes or reduce the number of shards per index.

What is the output of:

GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v

If some outputs are too big, please share them on and link them here.

BTW did you look at and ?

Cloud by elastic is one way to have access to all features, all managed by us. Think about what is there yet like Security, Monitoring, Reporting, SQL, Canvas, APM, Logs UI, Infra UI, SIEM, Maps UI, AppSearch and what is coming next :slight_smile: ...

Hi @dadoonet
thanks for ur quick reply, these the output of the mentioned commands

      "name" : "4ad0667b83bbe4b0b46d9c687fff253f",
      "cluster_name" : "753188385451:geophy-logssystem",
      "cluster_uuid" : "Dcyeu21UT1S7fL7weR1LBw",
      "version" : {
    "number" : "7.1.1",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "7a013de",
    "build_date" : "2019-10-08T13:47:09.433683Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
      "tagline" : "You Know, for Search"

    GET /_cat/nodes?v
    ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
    x.x.x.x            5          81   1    0.00    0.17     0.41 mdi       -      4ad0667b83bbe4b0b46d9c687fff253f
    x.x.x.x            7          81   3    0.29    0.32     0.55 mdi       *      542a9d72469d4dd9373b0099d18c4a2f

GET /_cat/health?v
epoch      timestamp cluster                        status discovered_master shards  pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1583325975 12:46:15  753188385451:xxxxxxxx green           2         2              true   2182 1091    0    0        0             0                  -                100.0%

for command GET /_cat/indices?v, all my indices in open status and green healthy

    health status index                                                                    uuid                   pri rep docs.count docs.deleted store.size
    green  open   index1                        XeOgZ3p3SF-VIBlL-OGB9A   5   1       5274            0      6.9mb          3.4mb
    green  open   index2                        Ynqo_kVyRaG_zhXgpf7I8w   5   1        102            0      273kb        139.6kb
    green  open   evra-prod-alb-logs-2020.02.21 aMkKlQqEQOaGzt3KBBkhQg   5   1      23387            0     46.8mb         23.4mb
    green  open   index3                        8V-QPY4TRsyCJpVDbPAo5w   5   1        508            0      3.1mb          1.5mb
    green  open   index4                        fW6SQdbgQFO5wGwi0l1HGg   5   1        270            0    424.5kb        212.2kb
    green  open   index5                        VsTizVaLTU68ovVVEfQ93w   5   1      12708            0     16.9mb          8.4mb
    green  open   index6                        abMOPfiNTrCRZ68bvGe0eA   5   1      70703            0     92.6mb         46.3mb
    green  open   index7                        8oJz97tUQ5-cKdzOQq0vRQ   5   1       3361            0      3.3mb          1.6mb
    green  open   index8                        buH2H7lmSNOiLHdOvjFnoA   5   1        576            0        1mb        513.5kb
    green  open   index9                        4JPdqg62SEGHp9KUtbOYaA   5   1       7695            0     13.4mb          6.7mb
    green  open   index10                       WsOR1c8GTuKSERk4kkfwEQ   5   1       1187            0      2.3mb          1.1mb

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:


This is the icon to use if you are not using markdown format:

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

sorry @dadoonet, just edited

I highly doubt of this last output...

I can see 50 primary shards here where you have reported in total 1091 primaries.

ys i didn't copy full output but all of them in started status

So I can just guess.

You probably have too many shards per node.

May I suggest you look at the following resources about sizing:


Thanks @dadoonet
sorry, how can i change the number of shared per index and is it recommended to change to monthly index instead of daily index ?

For existing indices the shrink API might be used.
For coming indices, create or modify the index template.

Monthly, yearly indices might be a good option. It depends on the index you have today. As you didn't want to share the full output, I can't really tell.

But you can also look at the rollover API.

sorrry @dadoonet. it's too long :smiley:

app2-prod-app-logs-2020.02.18                                         1 r STARTED  1534  587.8kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.18                                         1 p STARTED  1534  587.8kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.18                                         2 p STARTED  1463    570kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.18                                         2 r STARTED  1463    570kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.18                                         4 p STARTED  1452  565.6kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.18                                         4 r STARTED  1452  565.6kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.18                                         3 r STARTED  1500  574.6kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.18                                         3 p STARTED  1500  574.6kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.18                                         0 p STARTED  1474  582.1kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.18                                         0 r STARTED  1474  582.1kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-elb-access-log-2020.03.04                                        1 p STARTED  1683    1.8mb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-elb-access-log-2020.03.04                                        1 r STARTED  1683    1.7mb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-elb-access-log-2020.03.04                                        2 r STARTED  1725    1.8mb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-elb-access-log-2020.03.04                                        2 p STARTED  1725    1.9mb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-elb-access-log-2020.03.04                                        4 r STARTED  1717      2mb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-elb-access-log-2020.03.04                                        4 p STARTED  1717      2mb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-elb-access-log-2020.03.04                                        3 p STARTED  1644    1.8mb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-elb-access-log-2020.03.04                                        3 r STARTED  1644    1.7mb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-elb-access-log-2020.03.04                                        0 r STARTED  1743      2mb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-elb-access-log-2020.03.04                                        0 p STARTED  1743      2mb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959

there is limit to past all there :frowning:

app2-prod-app-logs-2020.02.13                                         1 p STARTED   215  162.3kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.13                                         1 r STARTED   215  162.3kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.13                                         2 r STARTED   150  125.1kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.13                                         2 p STARTED   150  125.1kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.13                                         4 r STARTED   168  134.6kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.13                                         4 p STARTED   168  134.6kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.13                                         3 p STARTED   183  144.6kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.13                                         3 r STARTED   183  144.6kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-app-logs-2020.02.13                                         0 r STARTED   194  150.8kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-app-logs-2020.02.13                                         0 p STARTED   194  150.8kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959

app1-prod-alb-logs-2020.02.15                                            1 r STARTED   777  916.7kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app1-prod-alb-logs-2020.02.15                                            1 p STARTED   777  916.7kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app1-prod-alb-logs-2020.02.15                                            2 p STARTED   724  910.5kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app1-prod-alb-logs-2020.02.15                                            2 r STARTED   724  910.5kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app1-prod-alb-logs-2020.02.15                                            4 p STARTED   754  732.8kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app1-prod-alb-logs-2020.02.15                                            4 r STARTED   754  732.8kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app1-prod-alb-logs-2020.02.15                                            3 r STARTED   767  913.1kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app1-prod-alb-logs-2020.02.15                                            3 p STARTED   767  913.1kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app1-prod-alb-logs-2020.02.15                                            0 p STARTED   750  821.5kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app1-prod-alb-logs-2020.02.15                                            0 r STARTED   750  821.5kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959

app2-prod-alb-logs-2020.02.16                                         1 r STARTED   796  742.3kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-alb-logs-2020.02.16                                         1 p STARTED   796  742.3kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-alb-logs-2020.02.16                                         2 p STARTED   769  823.8kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-alb-logs-2020.02.16                                         2 r STARTED   769  823.8kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-alb-logs-2020.02.16                                         4 p STARTED   737  820.6kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-alb-logs-2020.02.16                                         4 r STARTED   737  820.6kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-alb-logs-2020.02.16                                         3 r STARTED   778  752.8kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-alb-logs-2020.02.16                                         3 p STARTED   778  752.8kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959
app2-prod-alb-logs-2020.02.16                                         0 p STARTED   725  729.9kb x.x.x.x 1688eee859e2418e813042d0b9f41f06
app2-prod-alb-logs-2020.02.16                                         0 r STARTED   725  729.9kb x.x.x.x  f9d895cc2b7d1a07911e23dd1ae22959

You have so small indices that only one index with one single shard would be probably enough.

You can share the output on as well.

I'm more interested by the following than the /_cat/shards

GET /_cat/indices?v

so i should set 1 shared to each index, right?

With the size of your indices, I think that one single index with one single shard is enough.

how can i set the default value for number of shared limit for coming indices. if i have different indices .

Also when i try to shrink the exist indices by shrink API "", got the below error

"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[f9d895cc2b7d1a07911e23dd1ae22959][x.x.x.x:9300][indices:admin/create]"}],"type":"illegal_argument_exception","reason":"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [2172]/[2000] maximum shards open;"},"status":400}[ec2-user@ip-172-20-1-235 ~]

i need to create target index to archive the exist ones

You can may be temporary change the current limit with

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.