Do you have the site running on localhost? Is it faster there?
Also do you have many IPs blocked? In 1.7 that can slow down the site a bit (will be fixed for 1.8).
Besides that it could be any number of things such as MySQL. Try changing the optimize options at the bottom of qa-config.php. If your MySQL server is on the same machine as the web server, set QA_OPTIMIZE_LOCAL_DB to true and QA_OPTIMIZE_DISTANT_DB to false. If the MySQL server is on a different machine, set the opposite (false and true).