Pagination with sorting - low number of users - 2-3mil documents

(Marek K) #1


I already asked about pagination of big ammount of data, but since last time I was trying to figure it out how I could reach my goal.

So I have only 500 users, which are going to make search requests. There will be 2-3 mil of documents.

In my opinion, number of users is not so big, 2-3 milions of documents in the world of indexing systems also seems to be not very impressive.

What I want to have is a pagination of sorted results. Sort will be done on _score and then on timestamp field.
Search will usually include 2, maybe 3 fields.

Everywhere I'm reading that this is bad idea to paginate sort results. I understand why it is a bad idea, but I have no idea how to resolve my problem. Scrolling may be an option because of small ammount of users, but this is not a real pagination.

So once again, I want to ask you, what could I read, where should I look for an answer for my question, how to implement properly pagination with sorting on score and timestamp fields?
I would like to add, that the documents will be changed only few days after creation of them. Documents which are ~1 week old, will not be changed, so maybe ES have special caching feature for such case?

(Nik Everett) #2

Deep pagination isn't a good idea - but you can absolutely get away with thousands of results. Even tens of thousands of results should be ok. It should work fine.

Scrolling might just work with that number of users but I'm not really sure - that you'd have to load test. Scrolling is trouble because it tries to give you a consistent snapshot at the cost of server side resources. Which is nice and maybe what you want it might have side effects that very few other users will see. Most people use scroll for administrative tasks....

(system) #3