There is no way to stop specific users from voting, but you can set a minimum points needed for voting. It can be below what users normally start with, then you can penalize that user by giving them negative bonus points.
There is also no way to remove votes on the site, except removing direct in the database. Find the userid then run: SELECT * FROM qa_uservotes WHERE userid=#
where # is the userid. Then click the button in admin>stats to recount posts/votes.
