Not sure what you mean by "friends" but no, the system doesn't prevent you from downvoting other answers. The points-based permissions in the next version should be able to stop new users downvoting if that answers your question?
I suppose it could be beneficial to block downvoting if a user has also answered the question, to stop people answering and downvoting others in order for their answer to rise to the top.
