Q2A doesn't have such a simple architecture that would allow only one file to modify a given section. For example, strictly answering your question, the file is qa-include/pages/user-answers.php, but if you are trying to edit CSS or add/remove information maybe the theme itself would be a better place to change (or creating a plugin, ideally).
Believe it or not, http://docs.question2answer.org/plugins and http://docs.question2answer.org/themes are the best sources of information.
EDIT
Based on your clarification that you just want to remove the All answers button from the user profiles except for Administrators you could do this:
1. Edit the file qa-include/qa-theme/<your-theme>/qa-theme.php
2. Add or merge the following initialize() function in the theme:
public function initialize() {
parent::initialize();
require_once QA_INCLUDE_DIR . 'app/users.php';
if (qa_get_logged_in_level() < QA_USER_LEVEL_ADMIN) {
if ($this->template === 'user-answers') {
qa_redirect('');
}
unset($this->content['navigation']['sub']['answers']);
}
}