Question about using Elasticsearch for RAG

Elastic Stack Elasticsearch
1

Just learned about the RAG Playground in Elastic, so I am hoping this is an opportunity to simplify my project:

Here is my scenario / goal:

We have an old rules-based app that parses store items like bikes, bike parts, etc. (It could be other products, and each product would have it's own domain/training)

Our historical data is already in elastic indexes - so we have a large dataset we can use to train against.

What I would like to do is replace the rules-based app with an AI-based process that would be more accurate, and ultimately faster.

So the workflow would essentially be -> read a piece of text or small json file that comes in as input -> LLM parses it into json using a defined schema for the product domain (using our existing indexes as trained data) -> then match it using elastic to a known catalog of items (also in elastic). if i could skip the "output it in a json format" and just go and match it against the catalog, that would probably be better but my thinking was there needs to be human review / training / prompt writing / rules updates that needs to happen to perfect the training/matching. I would l have a custom built “workbench” for this using elastic api’s.

Would love thoughts/feedback and some guidance about how to get started w/this.

Thanks,
Bob

2

Hi!

Let me start with some high-level guidelines -

  1. All RAGs need to also still have solid rule-based logic in the backend, one way or another. Especially if it was carefully curated over the years. RAG / GenAI is not magic - it just empowers you to do many more things on top of a solid baseline.
  2. LLM parsing should really just help you with extracting meaning, perform some routing, and potentially do query expansion. I wouldn’t necessarily ask it to write the Elasticsearch query DSL for me - but rather use it to extract keywords (additional ones, usually) or possible tags or properties to include or boost by.
  3. You mentioned training - you don’t really need to train anything here. You should either vectorize it and use semantic / hybrid search on it ; or use LLMs to explode your query with more meaning and use lexical search on it.
  4. You will need good evaluation / testing mechanism, that’s for sure.

Here are some good resources to start with (they mentioned OpenSearch but it’s just the same for Elasticsearch and any framework / workbench on top of it) :