# RAG - how is it used? - Context window - does it work out out of the box?

**URL:** <https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [March 8, 2024, 11:33am UTC](https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028 "2024-03-08T11:33:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chenko](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chenko](https://discuss.elastic.co/u/Chenko)\
**Post date:** [March 8, 2024, 11:33am UTC](https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028/1 "2024-03-08T11:33:57Z")

</div>

I am wondering how RAG is used, I see it has a context window in which it adds context, however what is this context?

Another questions I am having is, does RAG work out of the box?  
if so how?  
if not, what would need to be changed/programmed in order to make this work?

---

<div class="post-metadata">

**Author:** ![iulia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iulia/32/124658_2.png) [@iulia](https://discuss.elastic.co/u/iulia)\
**Post date:** [March 12, 2024, 10:20am UTC](https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028/2 "2024-03-12T10:20:27Z")

</div>

Hi!

I'll go over a general overview of how it works and link some articles to get your started.

RAG stands for Retrieval augmented generation.  
It takes the concept of:

- Generative AI - where a model (usually an LLM) takes in a question and generates an answer for it.
- Information Retrieval - searching for relevant documents within your database/index based on a Query.

Putting those together, when a user asks a question, you first retrieve relevant information that might help answer that question; then you give this information (maybe data entries from your database, some recent articles or news, etc) as context to the LLM.

So now you can generate an answer based on not only the initial question; but the additional information you retrieved as well - giving more accurate responses.

Check out this blog for full definitions and breakdown: [What is Retrieval Augmented Generation (RAG)? | A Comprehensive RAG Guide | Elastic](https://www.elastic.co/what-is/retrieval-augmented-generation)

As for if it works out of the box; you need to make some choices for your setup, but you can use out of the box components. You can choose which LLM to use for the generative part; which search engine setup to use for the information retrieval; and how you're going to put it together / serve it to the user.

Here is an example with Elasticsearch, Cohere, and Amazon Bedrock: [Retrieval Augmented Generation using Cohere Command model through Amazon Bedrock and domain data in Elasticsearch — Elastic Search Labs](https://www.elastic.co/search-labs/blog/articles/gen-ai-using-cohere-llm)

---

<div class="post-metadata">

**Author:** ![Chenko](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chenko](https://discuss.elastic.co/u/Chenko)\
**Post date:** [March 12, 2024, 10:35am UTC](https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028/3 "2024-03-12T10:35:22Z")

</div>

Thanks for the informational response!

I just have one more question, to be clear.  
RAG just does the following?

1. User types in a query
2. ES call (With vectorsearch)
3. Use the top X documents as Context for the LLM
4. send an api call to the LLM with the users original question and the received documents
5. LLM returns its response

Is that it/is that all?

---

<div class="post-metadata">

**Author:** ![iulia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iulia/32/124658_2.png) [@iulia](https://discuss.elastic.co/u/iulia)\
**Post date:** [March 12, 2024, 11:18am UTC](https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028/4 "2024-03-12T11:18:15Z")

</div>

In essence yes. Just to make it clear: RAG is a framework / concept not an actual tool. So as long as you're implementing those steps in some way you are indeed performing RAG.

---

<div class="post-metadata">

**Author:** ![Chenko](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chenko](https://discuss.elastic.co/u/Chenko)\
**Post date:** [March 12, 2024, 11:30am UTC](https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028/5 "2024-03-12T11:30:13Z")

</div>

Perfect, thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 9, 2024, 11:30am UTC](https://discuss.elastic.co/t/rag-how-is-it-used-context-window-does-it-work-out-out-of-the-box/355028/6 "2024-04-09T11:30:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.