updateByQuery cluster state red

Steps to reproduce:

  1. insert 45 000 000 documents
  2. run:
    let res = await client.updateByQuery({ index: config.get('index'), type: config.get('type'), slices: 5, waitForCompletion: false, conflicts: "proceed", requestTimeout: 36000000, timeout: -1, body: { "query": { "bool": { "must": { "match_all": {} }, "filter": { "bool": { "must": [ { "exists": { "field": "offers" } } ] } } } }, "script": { "lang": "painless", "inline": 'for (int i = 0; i < ctx._source.offers.length; i++) {ctx._source.offers[i].price_usd = ctx._source.offers[i].price / params.toUSD[ctx._source.offers[i].code]}', "params": { "toUSD": params } } } })
  3. After a while the cluster status will become red

Is it possible to update all records in index?

Why does the cluster go red? Are there any errors or warnings in the Elasticsearch logs? What does your documents look like?

We've just found the problem on one of our nodes.
No space left on device
During the updating elasticsearch eat a lot of space on disk. May be you know what can we do with it?

As Elasticsearch is working on of a point-in-time snapshot when the update by query operation starts, this dataset needs to remain, even if every document gets replaced. So if your update query operation affects a lot of documents, you need to have in the worst case twice the amount of index size available on disk.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.