Following on the 2009 Convention’s “Focus on Translation,” the MLA has just released an official statement on Evaluating Translations as Scholarship.
Overall, this is a pretty interesting document, both because it helps establish some guidelines for assessing translations in “personnel decisions related to hiring, retention, merit awards, promotion, and tenure.” Seeing that the general line for the past XX years has been that translation could hurt your chances at getting tenure, this is a pretty significant sea change.
The document Chad links to also has a lengthy section offering advice for reviewers of literary translations. Obviously, if this is something you do or want to do, you should read it.
Incidentally, Words Without Borders has been running a series of articles where various people opine on best practices in this area. I’m told my entry will go online soon.