Yes, the errors you get when you switch to XHTML are caused by Q2A's HTML tags being generated in upper-case, which is not acceptable XHTML.
The first two errors you are getting now could be solved by putting the required type="text/javascript" attributes on the two <script> tags introduced by the Facebook login plugin. You can change this inside the plugin PHP.
The next two are trickier. If you try to tell the validator about the FB namespace you will get another error. The best solution might be to get rid of this (again in the plugin):
<fb:login-button perms="email,user_about_me,user_location,user_website"></fb:login-button>
and replace it with this:
<script type="text/javascript">document.write('<fb:login-button perms="email,user_about_me,user_location,user_website"></fb:login-button>');
</script>
Since the FB connect code needs javascript to run, you're not losing anything by doing this.
Alternatively, if you are using jQuery with your theme (or in future using Q2A 1.4, which provides jQuery), you might find this sort of solution appropriate:
