Multi search API: Getting more than 500 hits

Hello guys!

I kindly need your help with something.

I'm building a script using groovy. The main goal is to retrieve all the logs we have on our server so I can use that logs, apply some logic (e.g., regex, conditional logic) and retrieve the results on a csv.

The thing is: in most cases when I try to use the Multi-search API, I only get 500 hits. On most case, the number is way larger than then (20k, for example).

I know that it's not possible to use a scroll mechanism on this API, but it's the only one I have access to (the platform in which we have our API's - Mulesoft -, only exposes the _msearch endpoint; it's impossible to use another one).

I can't really work only with 500 hits.
My goal is to use Groovy/Java to make this happen and I can't come up with a solution.

Do you guys have any idea of how I can achieve this result to retrieve all the information? Some kind of logic that allows us me to say to the API: Ok, find me the first 500, then go ahead and retrive the next 500, and so on.

Thank you!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.