Hi,
I think that I know the cause of your problem:
The search request has a "size" parameter that sets the number of hits to return Search request parameters .
If omitted, the default value is 10.
You can modify this parameter (see the example in x-pack input search to get only the last result)
Unfortunately, I don't know any elegant, scalable solution other than setting a suitable magic number .
After reading this question: Watcher index action - how to get all the hits data to index it seems that:
- The limit is 10000
- It is not a good idea to index a big amount of search results.