# Question about using Elasticsearch for RAG

**URL:** <https://discuss.elastic.co/t/question-about-using-elasticsearch-for-rag/383993>\
**Category:** Elasticsearch\
**Created:** [December 11, 2025, 4:18pm UTC](https://discuss.elastic.co/t/question-about-using-elasticsearch-for-rag/383993 "2025-12-11T16:18:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![bobwiller](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@bobwiller](https://discuss.elastic.co/u/bobwiller)\
**Post date:** [December 11, 2025, 4:18pm UTC](https://discuss.elastic.co/t/question-about-using-elasticsearch-for-rag/383993/1 "2025-12-11T16:18:39Z")

</div>

Just learned about the RAG Playground in Elastic, so I am hoping this is an opportunity to simplify my project:

Here is my scenario / goal:

We have an old rules-based app that parses store items like bikes, bike parts, etc. (It could be other products, and each product would have it's own domain/training)

Our historical data is already in elastic indexes - so we have a large dataset we can use to train against.

What I would like to do is replace the rules-based app with an AI-based process that would be more accurate, and ultimately faster.

So the workflow would essentially be -\> read a piece of text or small json file that comes in as input -\> LLM parses it into json using a defined schema for the product domain (using our existing indexes as trained data) -\> then match it using elastic to a known catalog of items (also in elastic). if i could skip the "output it in a json format" and just go and match it against the catalog, that would probably be better but my thinking was there needs to be human review / training / prompt writing / rules updates that needs to happen to perfect the training/matching. I would l have a custom built “workbench” for this using elastic api’s.

Would love thoughts/feedback and some guidance about how to get started w/this.

Thanks,  
Bob

---

<div class="post-metadata">

**Author:** ![Itamar\_Syn-Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn-hershko/32/118620_2.png) [@Itamar\_Syn-Hershko](https://discuss.elastic.co/u/Itamar_Syn-Hershko)\
**Post date:** [December 31, 2025, 6:37am UTC](https://discuss.elastic.co/t/question-about-using-elasticsearch-for-rag/383993/2 "2025-12-31T06:37:28Z")

</div>

Hi!

Let me start with some high-level guidelines -

1. All RAGs need to also still have solid rule-based logic in the backend, one way or another. Especially if it was carefully curated over the years. RAG / GenAI is not magic - it just empowers you to do many more things on top of a solid baseline.
2. LLM parsing should really just help you with extracting meaning, perform some routing, and potentially do query expansion. I wouldn’t necessarily ask it to write the Elasticsearch query DSL for me - but rather use it to extract keywords (additional ones, usually) or possible tags or properties to include or boost by.
3. You mentioned training - you don’t really need to train anything here. You should either vectorize it and use semantic / hybrid search on it ; or use LLMs to explode your query with more meaning and use lexical search on it.
4. You will need good evaluation / testing mechanism, that’s for sure.

Here are some good resources to start with (they mentioned OpenSearch but it’s just the same for Elasticsearch and any framework / workbench on top of it) :

> **[RAG with OpenSearch: A Practical Guide to Building Production-Ready AI...](https://bigdataboutique.com/blog/rag-with-opensearch-a-practical-guide-to-building-production-ready-ai-applications-8dcce4)**
>
> In a recent webinar we showed how to build production-ready RAG applications with OpenSearch. Complete guide covering data processing, hybrid search, and LLM integration for AI chatbots and context-aware systems.

> **[Semantic Search Application with Elasticsearch Vector Database](https://bigdataboutique.com/blog/semantic-search-application-with-elasticsearch-vector-database-ceed7f)**
>
> A hands-on tutorial for building a semantic hybrid search application with Elasticsearch as a vector database.

> **[Innovating Search Experience with Amazon OpenSearch and Amazon Bedrock](https://bigdataboutique.com/blog/innovating-search-experience-with-amazon-opensearch-and-amazon-bedrock-d045bc)**
>
> In today’s competitive digital marketplace, delivering a great search experience is key. This blog shares how we used Amazon OpenSearch and Amazon Bedrock to enhance a customer’s search functionality, including using a large language model (LLM) to...