evaluation - How should we evaluate our peers?

This is a quite broad question, to which I have some personal answers, but I think I need more different perspectives.

I am not talking about evaluating a peer's submitted work as a referee, but about evaluating applications ranging from small grants or sabbaticals (which have to be applied to at least in France) to tenure or tenure-track hiring. Such decisions have a small or huge impact on both others career and the overall academic system (by giving certain incentives), and should thus be taken very seriously. On the other hand, the time we are able to use for such evaluations is limited.

How should we deal with this opposing constraints (evaluation is serious matter but time is limited)? What tools and proxys should we use or avoid in the evaluation process?

I am not asking about how the system should be (I know my answer: more automatic funding, sabbaticals and pay raises to have less evaluations made more thoroughly), but about what one wanting to take evaluation seriously (as opposed to a formal game that only needs an arbitrary answer) can do in the current system. Actions that might imply change in the system are welcome, as far as they can be implemented individually and the plausible outcome is considered rather than the ideal outcome if everyone did the same thing.

Edit in view of comments

Clarification about "the system": if needed please precise in your answer the relevant bits of information about the evaluation system you are speaking about, as of course things evolve and vary from place to place -- but the broad principles should be applicable regardless of the fine details so please keep this as concise as is relevant.

Example of subquestions: it has often been said that impact factor, or more generally the glamor of journal titles should not be used for evaluation. Is there a consensus about this? I don't know. They are used in practice, but are there good alternatives? Are there specific evaluation circumstances when one should use this proxy?

What about h-index and other citation metrics? When should they and should they not be evaluated?

The question is not so much about how much time one should spend (which is more or less a given), but how to achieve a reasonable result in that time. Also, what to do in a situation where one feels a reasonable result cannot be achieved?