Updating non-file synonyms

[Henr1k (Henrik Gedionsen)]

Hi, I was trying to do updates on non-file synonyms, but it fails.
Do I have to use a file or am I doing it wrong?
Files isn't that well supported in Elasticsearch Cloud.
It is only via Cloud UI or have I missed an API way of uploading a new file?
I know I can close and open the index, but I would like to avoid downtime by just reloading analyzers like mentioned here https://www.elastic.co/blog/boosting-the-power-of-elasticsearch-with-synonyms.
Steps to recreate:

PUT /test_index
{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "synonym": {
            "type": "synonym_graph",
            "updateable": "true",
            "lenient": "true",
            "synonyms": [
              "lol, laughing out loud",
              "universe, cosmos"
            ]
          }
        }
      }
    }
  }
}

PUT /test_index/_settings?pretty=true&error_trace=true&preserve_existing=true
{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "synonym": {
            "synonym_graph": {
              "synonyms": [
                "lol, laughing out loud",
                "universe, cosmos",
                "i-pod, i pod => ipod"
              ]
            }
          }
        }
      }
    }
  }
}

fails with
Can't update non dynamic settings [[index.analysis.filter.synonym.synonym_graph.synonyms]] for open indices

[Jaspreet_Singh]

When you update synonyms, you need to make sure existing data in your indexes is re-processed as per new synonyms.
One way that takes care of update without downtime and re-process existing data may be to create a new index with new mapping that includes your inline synonyms.
This is followed by using _reindex API to bring data from old index to new.
Finally, you flip the alias from old to new index.

[cbuescher (Christoph)]

Hi @Henr1k,

as you already found out, managing updateble synonyms in-line in the index settings is not possible at the moment and there are no real plans to do so in the future. Instead, we hope to make managing synonyms easier with a dedicated API (see https://github.com/elastic/elasticsearch/issues/38523), however current efforts with that regard have been stalled for a while.
You can see how to upload resources like synonym files to Cloud here I believe: https://github.com/elastic/elasticsearch/issues/38523, however after reading it I'm not 100% sure if uploading a new synonym file as a bundle like described there will make it visible to already running nodes or if there is an alternative cloud API to do that. I would suggest also asking in the cloud forum around here about that since I'm not so knowledgeable around that topic tbh.

Regards

[Henr1k (Henrik Gedionsen)]

Hi @cbuescher,
Ok, thank you.
There is a way to update an extension (with the synonym file) in the could API Deployments - Extensions | Elasticsearch Service Documentation | Elastic
But it seems like extensions are only updated on deployment startup.. Upload custom plugins and bundles | Elasticsearch Service Documentation | Elastic

These are downloaded and made available when a node is started.

I guess that restart of the deployment is worse than opening and closing the index...

@Jaspreet_Singh thank you but, the reindex is not viable for me, I would like a more realtime way of changing synonyms and not reindexing for every change.

[Jaspreet_Singh]

@Henr1k Right - having a synonym file is the way to go.
But I'm curious, how do you plan to re-process already indexed content as per new synonyms. By default only new content that is ingested will be processed as per new syonyms.

[Henr1k (Henrik Gedionsen)]

@Jaspreet_Singh I prefer to apply synonyms at query time, not index time. In fact, that is the only way possible with the synonym_graph filter

[Jaspreet_Singh]

oooh! I see That explains