Too many indices

(Gabriel Asherov) #1

Hello everyone.
I am new to elasticsearch and don't really understand the documentation for nodes and shards and all of that stuff. My problem is that I have around 2-3k indices and when I try to create an index pattern in kibana or basically whenever I try to do anything in kibana it just freezes and then crashes, giving me an error 503 "all shards failed". After researching a lot I think the problem is that I have to many shards(?)
As I said I just don't understand it that well.
I hope someone can help me.

(Christian Dahlqvist) #2

It sounds like that may very well be the case. I would recommend reading this blog post about shards and sharding for guidance.

(Gabriel Asherov) #3

Okay so I just read through all of that but I still don't get how I can have 2000 indices and not have performance issues. I need an index for every customer and there is just one document every day per customer. So I'm guessing I don't need 5 shards + replicas for each index (?)
And if so, how do I tell elasticsearch to use less shards per index?
I'm running elasticsearch in a docker container by the way.

(Christian Dahlqvist) #4

Having an index per customer tends to scale badly. Why do you not have the customers share an index and add a field specifying the user that you can filter on?