Hmm, I'm a bit confused?
Do you want to show the top 50 IDs, and the most recent document for each ID? If this is the case, you can just use the Terms agg, collect all the results, and apply pagination in your application.
Or do you want to show the top 50 IDs, and show a list of documents for each ID, sorted by timestamp? In that case, I would just execute a separate search to get those documents. E.g. use the Terms aggregation to determine your top IDs to show, then execute a separate search for each of your top IDs and sort that search by time descending. This will allow you to paginate.
There's not much we can do about _source if it's gone. You'll have to fetch the document from your other document store. If ES doesn't have it stored...it doesn't have it stored.