How to increase the speed of response?

Elastic Stack Elasticsearch
1

Hi
I have a lot of data up to 10 billion and in order to be quick I made 7 contracts and it was still very slow
What is the best practice in order to be a quick response?

1 Like
2

What do you mean?
Any example of a request and a response?

3

I mean that I have a lot of data that can be increased very quickly so how can I make it very fast
What do you advise me to be quick

For example
Suppose I have data the size of Netflix data, what is the best practice to be fast?

4

I do not have a specific request, it is very slow in all requests

5

I use the following search:
Query string
Match
Match all
Term
Wildcard

6

Wildcard queries are generally one of, if not the, slowest type of query in Elasticsearch (especially leading wildcard clauses) so based on the information you have provided I suspect that is at least part of your problem. Try removing any wildcard query clauses and see how this affects latency.

If you can get around the need for wildcard queries, index your data using the wildcard field type and use this for wildcard queries.

7

It would help to show examples of requests and responses.
Also what is your version? What is the hardware used? How many nodes? How many shards????

1 Like
8

I need it so much.I need it so much.

9

In general, all requests are very slow, the response time is high.

For example I want to fetch data provided that there are 3 fields that have specific values

Version : 8.15.1
Devices used : ZBOOK
RAM:64
Number of nodes : 7

10

Are you running 7 nodes on a single laptop? If so, why? Does it have an SSD?

How many shards are you querying? What is the total size of all shards? What is the average shard size?

11

No
I am running 7 nodes on 7 different environments

12

Do the nodes have SSD?

How many shards are you querying? What is the total size of all shards? What is the average shard size?

13

Yes, it contains an SSD and at the same time HDD, I save the data on the HDD, but elastic is installed on the SSD disk, but I change the data.baht to the HDD disk

As for the size of the fragments, it is 1 per index, and the data is divided into 30 indexes for each index, one fragment

14

size of all shards 5TB

16

This means that the Elasticsearch data sits on the HDD which tend to have very limited I/O performance. As you can see from the official guide on tuning for search speed it is recommended to use SSD for the Elasticsearch data directory.

Is that total size or size per shard?

17

Yes, all nodes contain an SSD, but the place to save the data is on HDD

18

No, this is the total size of all fragments, and it is not only increasing on a daily basis.

19

It does not help.
Unless you are saying that running the following request is super slow:

GET /index/_search
20

I would recommend the following:

  • Move the Elasticsearch data directory to SSD on all nodes.
  • Provide the full output of the cat shards API.
  • Show your mappings as well as a sample query that is slow.
  • If you have only 5GB of data spread across 30 indices, reindex the data into a single index and see what impact on latency that has.
  • Remove the wildcard clause/clauses and see if that impacts latency.
21

Well, I'm going to try that out.

Thank you