Linus Dahlander, professor of strategy and Lufthansa Group Chair of Innovation at ESMT Berlin, alongside colleagues from Purdue University and INSEAD, investigated the impact of peer evaluations on the behaviors of Wikipedia members, for which peer evaluations are transparent.

Peers can see the complete evaluation history of a member, including how and whom they have evaluated in the past, and these peer evaluations are used to determine which members become administrators.

The researchers focused on three key factors: whether the member was about to be evaluated themselves, how pivotal an evaluation was, and the candidate’s activity level.

Their research revealed that members who are about to be evaluated themselves participate in more peer evaluations. However, members are less likely to participate in evaluations if their evaluation may offend someone or be pivotal in impacting a peer’s overall assessment, focusing their negative evaluations on inactive members. Negative evaluations are also targeted toward those that are unlikely to swing the evaluation outcome in either direction, in which the overall outcome is already obvious.

The research also found no evidence that members focus on giving positive evaluations to active peers, suggesting they avoid negative reciprocity but do not attempt to invoke positive reciprocity. Further analysis suggests that this strategic use of peer evaluations is effective, with members more likely to be evaluated positively and to get promoted by their peers.

“Our research shows people tend to participate in peer evaluations if they believe it will benefit them rather than if their evaluation would be helpful. This means they refrain from participating in evaluations where the outcome is uncertain to avoid retaliation. Incidentally, the organization is likely to miss important evaluations when they could be most valuable,” says Prof. Dahlander.

The findings demonstrate that, although transparency and self-selection make evaluations more accountable, it allows members to use their evaluations to portray themselves strategically ahead of their own evaluation.

To reduce the opportunity for strategic manipulation, organizations should implement transparent peer evaluation processes with clear guidelines. While transparency can enhance accountability, managers should be aware that employees might use this transparency to their advantage strategically.

Organizations can hold members accountable and foster a culture of genuine merit-based assessments by ensuring evaluations are open and traceable. This approach can enhance trust in the evaluation system and improve organizational fairness and effectiveness.

Managers should also encourage employees to provide balanced evaluations that reflect both positive and negative aspects of performance, irrespective of personal stakes. Training programs on effective feedback delivery and the importance of objective evaluations can help mitigate the strategic biases identified in this study.

This research was published in the journal Organization Science and may be viewed here.