Query for large numbers

[tomer (tomer zaks)]

Hi I am trying to query for around 500K transactions per 5 minutes therefor I did:

curl -XGET '192.168.1.116:9200/filebeat-*/log/_search?pretty&size=500000' -H 'Content-Type: application/json' -d'

but I get null... when I do:

curl -XGET '192.168.1.116:9200/filebeat-*/log/_search?pretty&size=10000' -H 'Content-Type: application/json' -d'

I get a correct answer.

How can I

[Christian_Dahlqvist]

Size is limited to 10000. To return that number of results you will need to use scroll query.

[tomer (tomer zaks)]

so if I use this query as follows:

"POST /twitter/tweet/_search?scroll=5m"

The query will be alive for 5 minutes and will gather all the info inside it for when started for 5 minutes?

[Christian_Dahlqvist]

No, you will need to use the scroll contexts to retrieve batch after batch through multiple requests. The scroll context will be kept alive for at most 5 minutes.

[tomer (tomer zaks)]

Thanks so I saw that by the "size" I decide how many hits are returned per "batch". Our product is not yet in production, therefore, I can't completely test this so maybe you can help me and tell me where do I see the total when there are more batches?

Now I can see them in the total term. I saw that also when I do size:5 I get it in the total even when there are more than five.

Is this the case for big numbers does this singular "total" term show the total?