Query for large numbers

Hi I am trying to query for around 500K transactions per 5 minutes therefor I did:

curl -XGET '*/log/_search?pretty&size=500000' -H 'Content-Type: application/json' -d'

but I get null... when I do:

curl -XGET '*/log/_search?pretty&size=10000' -H 'Content-Type: application/json' -d'

I get a correct answer.

How can I

Size is limited to 10000. To return that number of results you will need to use scroll query.

so if I use this query as follows:

"POST /twitter/tweet/_search?scroll=5m"

The query will be alive for 5 minutes and will gather all the info inside it for when started for 5 minutes?

No, you will need to use the scroll contexts to retrieve batch after batch through multiple requests. The scroll context will be kept alive for at most 5 minutes.

Thanks so I saw that by the "size" I decide how many hits are returned per "batch". Our product is not yet in production, therefore, I can't completely test this so maybe you can help me and tell me where do I see the total when there are more batches?

Now I can see them in the total term. I saw that also when I do size:5 I get it in the total even when there are more than five.

Is this the case for big numbers does this singular "total" term show the total?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.