As you said, we are only hiding the voting buttons.But the count will be shown..Just replace the function in qa-theme-base.php
Edit: You should add this function to your theme's qa-theme.php file.So it will not affect future updates.
public function voting_inner_html($post)
{
if (qa_is_logged_in()) {
$this->vote_buttons($post);
}
$this->vote_count($post);
$this->vote_clear();
}
