There isn't currently an admin setting for this, but you can fix it by changing this line in qa-page-categories.php:
$qa_content['ranking']=array('items' => array(), 'rows' => count($categories));
... to ...
$qa_content['ranking']=array('items' => array(), 'rows' => ceil(count($categories)/2));
Replace the 2 above with the number of columns you want.