Response truncated in GenAI playground

Elastic Search
1

I'm trying out the Playground tech preview with ELSER and Gemini 1.5 Pro on my GCP-based Elastic Cloud cluster. This is so powerful and almost idiot proof to set up. I understand it's not officially supported, but I'm close to getting a decent demo and wonder if I could get some pointers getting it across the line.

  1. The generated response is often truncated midway through the Summary and without any Citations. I'm only getting about 800-1000 characters. What governs this and how can I increase it?
  2. I often get "Unterminated string in JSON at position 1175" as the response. Any idea what causes this or how to work around it?
1 Like
2

Hi there, glad you're getting good results with it.

  1. Would it be possible to try out a different model (for example OpenAi's) just to double check that the issues are with the model vs playground itself. We will also check our side on Gemini.
  2. We re-write the question based on the conversation for better RAG retrieval and gemini may struggle with this task. Claude and Openai likely give better results here
3

Hmmm, I'm not sure I can try a different model. I'm restricted to using GCP-based Elastic Cloud deployments, and inside of that I only appear to have Gemini 1.5 Pro or Flash as options.
My hunch is that the issue isn't with Playground, as sometimes (1 in 10) I will get an un-truncated response with both a Summary and Citations section.

Perhaps Gemini 2.0 will help here, if that's in the offing for Playground. If you'd like someone to alpha test it with real-world data, I would be thrilled to help. :wink: