In the "qa-content/qa-page.js" file, find (arround line 25)
function qa_reveal(elem, type, callback)
{
if (elem)
$(elem).slideDown(400, callback);
}
replace with
function qa_reveal(elem, type, callback)
{
if (elem)
$(elem).slideDown(400, callback).delay(2000).slideUp(400);
}
This should do about what you are looking for (in this case the "time out" for the notice display is set to 2 sec (2000 ms).
However, this is a hack in the core, not a possibility to set it via the admin panel as you suggest