Suggestion payload is not updated on document update

Elastic Stack Elasticsearch
1

Hi,

I'm having a problem with suggestions payloads that are not being updated with the main document.

It goes as follows:

  1. I create a document with the Java client which contains completion suggester data including a payload.
  2. I update the payload of the suggest data.
  3. I issue a suggest query but what I see is the old payload.
  4. I optimize the index with _optimize?max_num_segments=1.
  5. I issue the same suggest again. This time I see the correct payload.

A very weird observation: When I create the document through the HTTP API via Sense and update the payload then the suggest index is updated immediately. However, when I do it trough the Java (actually Scala) client then it is not. My hypothesis is that Java is using some binary API to do indexes and updates and that there are some subtle differences there.

I'm working on a case to reproduce this but I'm wondering if anyone has experienced something like this before.

It seems to be somewhat related to AutoCompletion Suggester - Duplicate record in suggestion return

Thanks for you help,
Leonard