Tips to increase query performance

[jarnor (Johan Arnör)]

Hi!

I'm wondering if you have any tips to increase the query performance of App Search. We've an index with ~10 million documents which have quite poor query performance. Simple queries can take between 4-5 seconds and paging between the documents in the App Search dashboard (/documents) takes >2 seconds.

This can be compared to ~300 ms (for paging in the dashboard) for another index which only have ~300k documents, but identical schema/data.

The query performance barley changed when moving from a 8 GB RAM to a 58 GB RAM cluster in Elastic Cloud.

I'm quite green on elasticsearch in general, but noticed that the index .app-search-engine-xxxxxx (the one powering the queries I assume) only is divided between two shards? Could this be related to the poor performance? Is there any possibility to increase the shard count?

Best,
Johan

[goodroot (Kellen Evan)]

Hey there, Johan -- welcome to the community.

Depending on which App Search version you are running, you may have access to the Result Setting feature.

The purpose of the feature is to help people refine their default query into something more neat and well-performing. While you may not have access to the feature, the rationale in the above guide can still be applied via your own custom API queries to ensure there isn't any undue strain placed on the system.

I'd be curious to see an example query -- do you have an example of one which is taking a long time? Or an example object (with anything sensitive removed!) which we could look at?

Thank you,

Kellen

[jarnor (Johan Arnör)]

Hi, and thanks for the reply.

We don't seem to have access the Result Setting page, we're using the latest self-managed version (7.4.0).

The documents in the index are quite lightweight, you can see a sample here - https://pastebin.com/3zivMgRE

Even the most basic queries are very slow, e.g. paging in the dashboard. However here is a heavier query which most of the time times out.

{ "query": "", "facets": { "item_status": { "type": "value", "size": 10 }, "brand": { "type": "value", "size": 10 }, "type": { "type": "value", "size": 10 }, "user": { "type": "value", "size": 10 }, "site": { "type": "value", "size": 10 }, "bag": { "type": "value", "size": 10 }, "container": { "type": "value", "size": 10 }, "photographed_by": { "type": "value", "size": 10 }, "is_fixed_price": { "type": "value", "size": 10 } }, "page": { "size": 100, "current": 1 } }

... and the response

{ "meta": { "warnings": [], "page": { "current": 1, "total_pages": 1, "total_results": 0, "size": 10 }, "alerts": [ { "code": 5001, "message": "Degraded search results", "link": "https://swiftype.com/documentation/app-search/alerts#5001" } ] }, "results": [] }

Do you have any recommendation about the cluster on Elastic Cloud? E.g. config or suitable cluster size that would be suitable for an index with ~10 million documents/60 GB

All the best,
Johan