Size parameter - how to check all stats in field

[_Sergey]

Hey community,

Can you advise how correctly use parameter "size". I faced with situation when I need to use size (e.g, for field user, where a lot of user_names are presented). When I use size: 500, I only will see 500 users. But for a big period of time (week+) there are more than 40k records in this field and if I use size even = 10 000 my watcher will take a lot of time to check this stats (of course I use additionally lots of aggs (cardinallity, I guess too heavy)) and sometime watcher just stack...

Is their a good explanation about "size" parameter to read about? Also can I avoid using size (does it mean check all records ?)

Cause in visualizations you need to use size by default ( if I use Terms there)

Thanks

[spinscale (Alexander Reelsen)]

Hey,

is this really a watcher question? In visualizations the use of size is different than in search query, because . a visualization is running an aggregation and not a search query.

Let me explain: Using size in a search request defines how many results should be returned back to the user. If you specify a size of 500, but your search returns 40k hits, then you will only see 500 documents in your search response. Also, it is not very performant if you want to return all those 40k hits at once using a regular search. This is was the scroll search is for.

However in many cases you are not interested in every single hit, but just a summary of the data, i.e. querying 10k records but you interest is only in a summary of the usernames, i.e. 500x the user name was jimmy and 1500 times the username was andrew. This is basically what the terms aggregation is for. You can also specify a size within the terms aggregation, which decides how many of those summaries should be returned. Aggregations support a size of 0 but there is still a safe guard that will not create more than 10000 buckets in the response. If you really need to paginate through all of your aggregation responses/buckets you can use the composite aggregation

hope this helps!

--Alex