Website search not returning consistent results

Elastic Stack Elasticsearch
1

Hi everyone,
I’m running a restaurant-based website that focuses on Texas Roadhouse menu updates, reviews, and nearby location listings. I’ve implemented Elasticsearch to power the internal search function on my site so users can look up dishes, restaurant branches, and coupons quickly. Everything was working fine initially, but over the past week, I’ve noticed that search results are becoming inconsistent and occasionally outdated. For example, when I update a menu item or add a new restaurant location, it doesn’t appear in search results right away — sometimes it takes several hours, and sometimes not at all.

I’m using Elasticsearch 8.14 on an Ubuntu 22.04 VPS. The website’s backend is built with Node.js and connects to Elasticsearch through the official client. Indexing happens automatically when new content is published through the CMS. However, when I check the cluster health, it shows as “yellow” most of the time, and the logs occasionally contain warnings about unassigned shards. I’m wondering if this could be causing the delay in document updates or partial failures in the indexing process.

Another issue is that some of the indexed fields seem to return partial matches or wrong results. For instance, when I search for “Texas Roadhouse ribs,” the top result is sometimes a blog post about “steak deals” instead of the actual ribs menu item. I’ve double-checked my mapping configuration, and I’m using the standard analyzer for most text fields. I suspect I might need a custom analyzer or a better query setup, but I’m not sure how to optimize that for restaurant and food-related content where names often overlap.

I also noticed a spike in resource usage when the indexing job runs. CPU usage on the server goes up to 90%, and the Elasticsearch process sometimes crashes under load. I’ve already tried increasing heap size and limiting batch sizes during indexing, but the issue persists. You can check my website https://thetexasroadhousemenu.com/. When the crash occurs, I occasionally lose partial data in the index, which might be contributing to the missing search results. My concern is that I might have a configuration issue with replica shards or memory allocation.

Additionally, I’m seeing slow query responses even for simple searches. Average search time has gone from under 100ms to over 1s. I’ve tried enabling query caching and optimizing field mappings, but it hasn’t made much of a difference. The /_cat/indices?v output shows a few indices growing unexpectedly large, possibly due to reindexing or duplicate entries. Could this be bloating performance and making search slower?

Has anyone else encountered similar issues where Elasticsearch becomes slow or unreliable with frequent content updates? What’s the best approach to keep my index fresh and search results accurate for a medium-traffic website like mine? Any guidance on optimizing mapping, analyzers, or shard configuration would be greatly appreciated. Sorry for the long post!

2

welcome back to the forum. This is 3rd thread you have started, over last year or two, all would broadly come under same “performance problems” heading.

This one has a bunch of red flags, just a few examples:

Any theory at all as to why?

That’s just bad, needs to be addressed.

For now I’d focus on stability of the cluster. Parts of your cluster are likely crashing regularly, you need to get to bottom of that first. IMO.

Is your cluster yellow right now, and if so do you know why? Do you know how to know why? You need also access the logs of each elasticsearch node and find out why it’s crashing, I am confident there will be logs. Out of memory (OOM) is a possibility, there are often system log entries for this too. Is the linux instance also crashing (check with uptime) or just the elasticsearch process (journalctl is your friend). Is the storage behind your nodes fast enough?

How are you monitoring your elasticsearch cluster? if something goes wrong, how do you get to know?

Do you make regular snapshots?

That’s uno, one, a singular instance on a singular VPS?

3

Is Elasticsearch the only component running on the host you are describing or is it shared with other components? If so, what is running on it and how much resources in terms of RAM and CPU are left for Elasticsearch?

What is the full output of the cluster stats API?

Is there a lot of iowait during indexing? What type of storage are you using?

What is the size and configuration of the node/cluster?

If you are using slow storage this might be an effect of the page cache not being properly used, resulting in slow disk reads. The amount of page cache and off-heap memory available to Elasticsearch is critical for optimal performance. Increasing the heap size does not necessarily improve performance unless the heap size is initially a limiting factor. If this is the case you can often see log messages indicating slow or frequent garbage collection.

This can be due to duplicates, but also possibly due to large number of updates and merging not being able to keep up due to slow storage. You should be able to see if merging is being throttled in the cluster stats I linked to earlier.