Limitation on number of docs for the Index action of a Watch?

Elastic Stack Elasticsearch
1

I cannot find this documented anywhere but is there a limitation to the number of documents that the Index action will process? We experienced two spikes in messages yesterday (798 and 3392) and in each case the watch action only indexed exactly 10 records on each run.

2

it's very likely that your query did not have a size set and thus only returned the first 10 hits.

You can verify this in the daily watch history index.

3

I can verify with 100% certainty that the query returned the correct number of hits.

            "condition": {
              "type": "compare",
              "status": "success",
              "met": true,
              "compare": {
                "resolved_values": {
                  "ctx.payload.hits.total": 798
                }
              }
            }
4

The hits itself are not returned, which are needed for bulk indexing. Just because the hit count is one million, it does not mean, that one millions hits/documents will be returned in the response. This is what the size parameter is for, which defaults to 10.

Showing the full watch history entry will show this.

1 Like
5

Of course. My bad. Thank you @spinscale

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.