Hmm, if understand right it is basically a speed problem ?
So, I think in this case limited paging lets say up to 10 pages max. would be fully sufficient. On heavily used sites it would still be not enough but more then 10 sites very most users wouldnt search.
Another way could be a dynamic solution where the number of sites depends on the dates only. lets say one could go back up to 4 weeks.
But this wouldnt help on big sites as the number of sites could become very huge.
But however, the all activities feature is very important as in a future release the community character would be supported very well so.
Actually I very few times use here on Your site the questions tab, If I pass by daily I perfectly see the new questions in the activity tab as well.
But good to know that You are having this in mind. Your script is really great.
And regarding the microformats I as well have a suggestion, but I will add it at the former question.