I'm building a Cache plugin/layer - What would you like to see it do?

Question

I'm building a cache plug-in layer for q2a. The basic idea is that, for most pages, the entire page can be cached, while other pages can only cache certain sections.

For pages like home that build elements from different places, I'm going to cache each major section as a cache file. When that part of the page is being built, I'll check if there's a cache file for it and use that. If there isn't I'll execute the code and store the result in a cache file to use next time.

For pages like a question, the entire page can be cached until there's a change event. To do this, I'm going to deliver the cached version of the entire page in the init of the modual if it's not a POST. If there's no cache file, I'm going to let the page execute and capture the output for the cache by turning on the output buffer with ob_start. If there's an event (like an upvote or comment) I'll delete the cache file so a new cache file could be created.

I'm going to store all cache information under the question ID so that I can wipe out the entire cache for that question, insuring the most up-to-date result each time.

Is this something you'd like to us? Please add your feedback and suggestions on this idea.

Edit: The existing version is very buggy. Use it for informational purposes only, not in a production environment. I'm working on an update that'll address a lot of the issues with this beta version.

Comments

Warning: This version has a lot of bugs that need to be addressed. Don't use it in a production environment.

---

This looks like a really useful add-on, but I'm wondering if it was ever really finished.  The Github page only shows 3 commits and none for the last year.  I assume then that this is not ready for production?

Answer 1

Just a word of warning, you won't be able to cache an entire question page, because  each user can have different buttons under each question/answer (edit, flag, comment) and they have a different user info box at the top.

Also, how will the plugin work in practice? Can you "intercept" Q2A at the start of a page load and prevent it from doing its usual thing?

Comments

The solution, for now, is to not cache data for users who are logged in. I'm expecting most of the traffic to come from people who are browsing and also search engines.

I already built the whole page cache engine (excluding logged in users) and it's working pretty well. I'm executing code within the qa-plugin.php file without waiting for the typical module load process. That means a lot of the base code hasn't executed and I have to do some URL parsing myself and load the qa-app-user.php file myself but that's very minor stuff. I'm seeing a large reduction in load time.

This also means I'm able to catch the final output and can save the server the compression time if it's been gzipped.

I included an event module to catch events with a prefix of q_ c_ a_ and assume it means new data was generated. At that point I delete the cache for that question.

My next step will be to intercept the build of other pages and try to feed it cached versions of things as it tries to build itself. For example, the homepage shows a bunch of Q&A's. I'd like to cache the details of each question and feed the code a cached version instead of it having to get it from the database. BUT I haven't looked at the code on how that page is being built and I might have to change this design.

Hopefully this can be done with a layer but I'll probably have to request some custom hooks be added to make this work. I'll know more when I dig deeper in the next few days.

Answer 2

I have a beta version ready for testing. Right now it caches full pages for anyone not logged in. It also keeps a gzip version at the ready so the server doesn't have to recreate that. If a question gets answered, edited or commented on, the entire cache expires. As I develop it more, I'll make the cache expiration more granular and only expire what needs to be.

https://github.com/MichaelApproved/Question2Answer-Cache-Plug-In

Edit: This version has a lot of bugs that need to be addressed. Don't use it in a production environment.

Comments

Good point. Right now I test to see if the user is currently logged in and look for the compression type. I don't check for the browser type or theme name. I'm going to experiment with themes and layers soon and, at that time, I'll have a look at how I can fix it. I'll post an update here when I come up with a fix.

In the meantime, I recommend disabling Cache That while using the theme switcher since it's not compatible with that layer.

---

This is a typical when page caching is involved, especially if the URLs are the same for both desktop and mobile. Maybe you can create groups of browsers and cache different versions of the pages. I don't think that creating one cached page per user-agent would be efficient since there are so many of them. You could either use .htaccess rewrite rules, or PHP, to serve from a particular directory.

I think that disk caching is awesome, but if the site gets super-busy (lots of Q&A), it won't be able to scale at some point, unless you leave the static page forever, until there is a change (why not?). Finally, if you are not using htaccess rules to serve the cached page, you could serve a stale page while rebuilding a new one. Thanks for sharing this!

---

Here's some great info from Stack Exchange on how they cache http://meta.stackoverflow.com/questions/69164/does-stack-overflow-use-caching-and-if-so-how

Eventually, it'd be great to cache parts of a page and complete the information using ajax. That would allow the server to deliver the same page for someone who's logged in as a non-logged in person. The logged in user info would be completed with an ajax request.

---

Great idea. Do you plan to continue your work?

Answer 3

I just posted another question that I realize only now could even be relevant for your cache plugin, as additonal feature.

thanks

Comments

If you build a layer that does a replace, the cache plug-in will serve that version without a problem. I'll comment more on your question thread.

Answer 4

Hi, I've also written a caching plugin, that serves pregenerated html files to unregistered users.

 

U can check it out here: https://github.com/bndr/q2a-caching

Comments

Hi bndr, are you using your plugin in production environment? Demo for speed?
I am very interested in caching and speeding up :)

---

Hi! Yes, i'm using it in production, but it's not user heavy (around 1000 users daily).

Sorry, no demo, but it does work as intended (i.e it serves static files without the burden of php execution or mysql queries for unregistered users). I would suggest you to turn it on for a few hours, see how it goes.