Elasticsearch not update

Elastic Stack Elasticsearch
1

I met a strange thing, that is, I can use _update_by_query locally to successfully update, but not on the server. I considered the problem of multithreading and added a lock, but it still doesn't work. Can anyone have idea?

2

I'm using 7.6.1 and default configuration, same on server.

3

Now I use nodejs client and got error online.

    2020-04-03T07:32:08.717Z { ResponseError: Response Error
    at IncomingMessage.response.on (/home/maimemo/status-server/node_modules/@elastic/elasticsearch/lib/Transport.js:296:25)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1129:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  name: 'ResponseError',
  meta:
   { body:
      { took: 2,
        timed_out: false,
        total: 13,
        updated: 0,
        deleted: 0,
        batches: 1,
        version_conflicts: 13,
        noops: 0,
        retries: [Object],
        throttled_millis: 0,
        requests_per_second: -1,
        throttled_until_millis: 0,
        failures: [Array] },
     statusCode: 409,
     headers:
      { 'content-type': 'application/json; charset=UTF-8',
        'content-length': '4921' },
     warnings: null,
     meta:
      { context: null,
        request: [Object],
        name: 'elasticsearch-js',
        connection: [Object],
        attempts: 0,
        aborted: false } } }
4

After debugging, I found some problems, because our update request is relatively fast, it seems that elasticsearch will not update.

POST /messages/_update_by_query?conflicts=proceed
{
  "query": {
    "term":{
      "roomId":"5d944ee17e78d618bec1a39b"
    }
   },
   "script":{"source":"ctx._source.status = 2"}
}

Return value begin is

{
    "took": 69,
    "timed_out": false,
    "total": 388,
    "updated": 388,
    "deleted": 0,
    "batches": 1,
    "version_conflicts": 0,
    "noops": 0,
    "retries": {
        "bulk": 0,
        "search": 0
    },
    "throttled_millis": 0,
    "requests_per_second": -1,
    "throttled_until_millis": 0,
    "failures": []
}

Change to

{
    "took": 22,
    "timed_out": false,
    "total": 388,
    "updated": 0,
    "deleted": 0,
    "batches": 1,
    "version_conflicts": 388,
    "noops": 0,
    "retries": {
        "bulk": 0,
        "search": 0
    },
    "throttled_millis": 0,
    "requests_per_second": -1,
    "throttled_until_millis": 0,
    "failures": []
}

updated is 0 . Will Elasticsearch not update under frequent updates?

5

Frequent updates to documents will cause a lot of small refreshes in newer versions which will lead to lower indexing performance as these are expensive and leads to a lot of merging. How often are you updating the same document?

6

The code is run in order and the two requests follow together. like this:

http.request()
http.request()
http.request()
http.request()
7

How frequently do you update a single document? Several times a second? A few times a minute? Do you always use update by query?

8

Yes, I always use update by query in a second.

9

I do not think Elasticsearch is optimized for frequent updates like this so would expect you to run into problems with this approach (although I have not tried it). It might be better to batch up updates in the application and send updates to Elasticsearch less frequently. If you describe your use case and what you are looking to achieve we may be able to suggest alternative approaches.

10

Thanks, I will try to reduce some useless requests to see if I can solve the problem. If it doesn't work, I'll come to consult again.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.