Deciding which new ideas are winners and which are duds is difficult for research and development (R&D) expert panels because of personal biases and the panel decision-making process itself, according to new research released in an article today by MIT Sloan Management Review.  

New research findings suggest that R&D expert panels are highly problematic for five reasons: 

• They show a strong bias against highly novel ideas. Decision makers often reject breakthrough innovations because they are uncomfortable with the risk pursuing them. They are more likely to fund projects with intermediate levels of novelty. Some degree of novelty increased the chance of funding, but too much reduced the odds.  

• They suffer from a lack of diversity. Decisions are largely based on the opinions of senior men who typically make up panels, missing valuable input from people with different experiences. 

• They’re staffed with scientist and engineers, who tend to focus on the technical aspects of an idea without considering the business opportunities and challenges. They may have a bias for ideas originating from their own area of expertise, and do not want a colleague’s success to surpass their own. 

• The panel decision-making process itself. R&D funding applications are often introduced by a panel member, who acts as its sponsor. Even if the sponsor strives to be objective, their own views on the project may be reflected in their tone and presentation, transferring biases and shaping views of others on the panel.   

• The timing of the process or the order in which projects are reviewed can influence decisions. Unique data from a professional services firm shows that the decision to fund one project makes it unlikely that the next project will be funded.  

“No one wants to be the decision maker who passes on a good investment,” says Thorsten Grohsjean, article co-author and assistant professor at Bocconi University in Milan. “Innovation will always be tough to assess, but by creating a process that is more open, fluid, and collaborative, organizations can spot the true gold nuggets among the rocks.”  

The MIT Sloan Management Review article, “Better Ways to Green-Light New Projects,” provides a number of recommendations for R&D directors and expert panels to reduce biases and improve outcomes before, during, and after selecting innovation projects for investment. These recommendations are illustrated with real-world examples from leading organizations such as Amazon, Bristol-Myers Squibb, the Defense Advanced Research Projects Agency (DARPA), BMW, Smart, Ericsson, New Zealand’s Health Research Council, Siemens, Shell, and Swiss National Science Foundation. 

Before projects are evaluated, companies can ensure they get a fair assessment based on their merits by revising the process for submitting ideas for consideration. To combat latent biases, organizations should remove names and demographic information about the creators of the new ideas, and standardize the information required in submissions so they are evaluated based on the same criteria. 

During the selection process, organizations should assemble a diverse selection panel in order to ensure that the viability of an idea is considered from a broader range of perspectives. Including people with both technical and non-technical backgrounds ensures projects are evaluated on technical aspects and also on market potential, business planning, strategic fit, and financing. Companies may also find it benecial to use crowd-sourcing principles, a workshop approach, leave it up to chance, or stage head-to-head comparisons. 

Once organizations have determined which projects to fund, they can provide feedback on proposals as well as track and learn from failures in order to help decision-makers make better choices the next time.  

“Although these practices may require some resources and effort to deploy, they likely cost less than the traditional model of selection, which heavily taxes senior managers’ and technologists’ time and effort,” says Linus Dahlander, article co-author and professor at ESMT Berlin. 

The article authors conducted a multiyear research project inside a global professional services firm with offices in 33 countries, collecting both quantitative and qualitative data about how they decide to invest in different innovations. In an initial study, the researchers examined how eight selection panels composed of senior managers and leading engineers allocate money to new research ideas. In a second study, they analyzed the effect of sequencing on 763 decisions made by one of the eight panels. The authors also conducted interviews with several dozen executives.

About MIT Sloan Management Review 

MIT Sloan Management Review (MIT SMR) is an independent, research-based magazine and digital platform for business leaders, published at the MIT Sloan School of Management. MIT SMR explores how leadership and management are transforming in a disruptive world. We help thoughtful leaders capture the exciting opportunities — and face down the challenges — created as technological, societal, and environmental forces reshape how organizations operate, compete, and create value.