Use an advanced theme:
http://www.question2answer.org/advanced.php#theme-advanced
In qa-theme.php in your theme directory, override the function a_count($post), something like this (code not tested):
function a_count($post)
{
if (@$post['answers']['data']>10)
$this->output_split($post['answers'], 'qa-a-count-many');
else if (@$post['answers']['data']>0)
qa_html_theme_base::a_count($post);
else
$this->output_split(@$post['answers'], 'qa-a-count-zero');
}
Then add some new CSS classes to qa-styles.css in your theme directory which match the HTML output for the different cases, changing the color as appropriate.