Slow search query issue [5.5]

Elastic Stack Elasticsearch
1

Dear Elastic community

We are struggling with very slow search querys. We have ~ 10 Mio documents and 140 Shards on one node. Elasticsearch is running on 7GB Ram and 2cores.

Each first request takes up to 15 seconds. Here an example request with profile information:

https://pastebin.com/FUv6rvT7

Here the result:

https://pastebin.com/jTiPMBce

Here the mapping:

https://pastebin.com/EvpzxB5y

1 Like
2

Hi, i'm also relative new to Elastic Search but i can already tell you that

140 Shards on one node.

sounds like a bad idea. Every shard is Lucene engine. So if your query needs to go though all 140 on one node might be that this is the problem and in result less fast, than having one, two or 3 shards.

3

We reduced shard size to 1 on the slow index. Before it was 5 and 5 unassigned. Now, with only 1 shard, it seems to be even slower than with 5 shards.

Sorry.

4

Ok 1 and 5 maybe makes no difference. But i was woundering bout 140...
I can't help you with the query, since i'm also just started with ES and have no experience

5

Any further suggestions?

Query is fast below 1 Mio documents. 1Mio+ its getting slower and slower.

6

So just a summary from our environment

Server:
24GB RAM
2 cores
100GB hdd

12GB RAM for elastic
all production settings were made like mlockall, etc.

13GB total documents in index
2 shards assigned for index

In favor of speed we removed the match for description in our query.

are there any values you need to know from kibana?
any suggestions what to investigate to find the issue

search speed, no matter what the input is under 3 mio documents ~20ms (constant)
search speed for 10 mio documents ~12000ms for the first query

afterwards (when elastic cached the search) it is around 500ms

I Hope anyone can help.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.