Cost Optimization with Generative AI Using Elasticsearch

Elastic Stack Elasticsearch
1

Hello Elastic Community,

I am conducting research on cost optimization strategies for generative AI systems. Specifically, I am exploring how Elasticsearch can be used to store user queries, model responses, and their vectorized representations. The goal is to minimize token consumption and reduce costs by retrieving cached responses for repeated queries and using generative AI models only for entirely new queries.

I would like to know:

  1. Are there examples of organizations successfully implementing such strategies with Elasticsearch?
  2. Are there specific challenges (e.g., scalability, accuracy) to be aware of when using Elasticsearch as a cache for generative AI?
  3. Any advice or best practices for implementing this architecture?

Thank you for your time and insights!

Best regards,