Download Link Sleuth and run it on your site
http://home.snafu.de/tilman/xenulink.html
You will get all the links in your site, create robots.txt based on that
On my yet to be published site, I finalized on the following
Disallow: /login Disallow: /ask Disallow: /forgot Disallow: /register Disallow: /questions?sort Disallow: /admin Disallow: /*?show=
Also I got the script listed here developed http://www.question2answer.org/qa/15139/fighting-google-panda
Of course, as you can see community did not like the idea showcased in my post but I stand by it as mentioned in the post
http://www.question2answer.org/qa/15139/fighting-google-panda
Welcome to the Q&A site for Question2Answer.
If you have a question about Q2A, please ask here, in English.
To report a bug, please create a new issue on Github or ask a question here with the bug tag.
If you just want to try Q2A, please use the demo site.