Define, broadcast, attract and select: a framework for crowdsourcing

Since its popularisation began nearly 15 years ago, internet crowdsourcing has gone from a leading-edge innovation practice to an almost obligatory technique. It’s no longer remarkable to see organisations ranging from local government agencies to major banks attempting to tap the “wisdom of the crowd” to generate new product ideas or business solutions.

By now, given all the available online tools for easy interaction, you would think crowdsourcing would be working wonders for firms across the board. Instead, crowdsourcing’s recent track record features many spectacular failures. Consider the example of crowdsourcing platform Quirky, which went bankrupt in 2015 despite raising US$185 million in venture capital. The company garnered stellar early press for its, well, quirky business model: developing product ideas submitted and selected by its 500,000-strong user base. Unfortunately, the crowdsourced concepts often had limited commercial appeal (Wi-Fi-enabled egg trays, anyone?). Regardless, Quirky had committed to putting resources behind whatever proposals the crowd chose, the prospective blockbusters and the bizarre alike.

In 2016, a similar “crowdsourcing fail” occurred when the United Kingdom’s Natural Environment Research Council (NERC) invited the public to choose the name of its newest polar research vessel. The moniker that won the online poll – Boaty McBoatface – drew the sort of global media attention that no serious government agency would want or appreciate.

The collapse of Quirky and l’affaire McBoatface demonstrate that “wisdom” is not the only quality crowds can exhibit. Crowds, after all, are composed of human beings. They can therefore display the same frustrating and foolish tendencies as individuals. The “wisdom of the crowd” turns out to be contingent. It results from a combination of well-aligned factors: the right crowd composition, presented with the right question at the right time, with the right analytic method applied to the responses.

To put it another way, think of crowd-based creativity as a natural resource. It’s not enough to get at it; you need to know how to harness it effectively and sustainably, as well as how to derive the most market value out of it. Just as oil companies don’t simply drill holes in the ground and hope for the best, companies should not attempt crowdsourcing without a solid framework to guide the project from inception to completion.

The “DBAS” framework

Based on a comprehensive review of the extant research (focused on organisation theory and innovation literature), we devised a crowdsourcing framework with four stages: Define, Broadcast, Attract and Select. (Our paper is forthcoming in Research in the Sociology of Organizations.)

Below, we describe all four stages in turn. You can also download a PDF detailing the framework.

First, you must know what you are looking for – we call this the Define stage. Is it a problem or a solution? How specific do you want your call-out to the crowd to be? Do you want to make one lump request, or is it better to break it up into constituent parts? Getting the question exactly right will help you spot irrelevant answers and weed out undesirables among the crowd.

Second, you must ensure that your crowdsourcing communication reaches the right people – which we call the Broadcast stage. Do you want to use your own platform or an intermediary? Do you want to convene a large crowd or a small one? How selective should you be in soliciting crowd contributions?

Third, to elicit the desired responses from the crowd, you must provide the proper motivation – the Attract stage. The three key decisions are:

• Should incentives be monetary?
• Should there be many winners, or few?
• Who owns the finished product?

Fourth and finally, there is the issue of choosing winners – the Select stage. Should you use judgment calls or a metric scale to evaluate entries? Should the crowd be involved in judging their peers’ work? How many rounds of judging should there be?

Obviously, each step along the Define-Broadcast-Attract-Select (or “DBAS”) pathway matters. But – and here’s the rub – how you navigate each stage can either reinforce or undercut decisions made at the other three stages too. From the initial stage of task definition onwards, companies need to cut a coordinated path all the way through the maze of decision making that crowdsourcing entails.

For example, properly navigating the Broadcast stage demands that the problem first be well defined, so that you can curate a crowd capable of supplying optimal solutions. Moving down the project pipeline, the Attract stage requires knowing what will motivate your crowd to exert the most effort – information that should be collected at the previous Broadcast stage. And during the Select stage, the optimal amount of resources committed will depend on the size of the contribution pool you have cultivated at Broadcast and Attract.

In addition, the DBAS framework can help you avoid some of crowdsourcing’s most common pitfalls.

Common pitfalls of crowdsourcing

Our ongoing investigations into the successes and failures of crowdsourcing employ huge data sets supplied by private providers of virtual engagement tools. The data track activity for tens of thousands of organisations that have installed online suggestion boxes. These data sets provide not only a complete, detailed, real-time view of crowdsourcing campaigns, but also a basis for comparing their results.

There are three major areas where a lack of forethought and careful coordination is hurting companies’ crowdsourcing efforts.

Innovativeness – Crowdsourcing is associated with creativity and innovation – the terms “crowdsourcing” and “open innovation” are often used interchangeably. However, they do not necessarily mean the same thing. Not all crowdsourcing campaigns require innovative and novel contributions. It is sometimes sufficient to take the pulse of a customer community or ask customers to choose between a small number of familiar options.

The question “How innovative do we need to be?” clearly belongs to the Define stage, but it has implications for the entire DBAS pathway. We have found that when crowdsourcing campaigns draw mountains of responses, the ones that are way outside the box are apt to go ignored. That is because if crowdsourcing evaluators feel overwhelmed by the volume of submissions, they are more likely to gravitate towards recognisable, eminently practical ideas and ignore outliers.

Therefore, if you want a high level of innovation from the crowd, you should take steps to constrain the number of submissions within an easily manageable range. At the Broadcast stage, innovation-seeking firms should be selective about whom they invite to participate, or build a few hurdles into the process to deter the least committed contributors.

Attention – Across all our data, there is a correlation between the amount of attention crowdsourcing campaigners give contributors, and the success of their initiatives. Two kinds of attention are salient here: reactive (offering feedback to external contributors) and proactive (organisational members submitting ideas themselves to kickstart the flow of ideas).

Campaigns fared better, in terms of both idea quality and quantity, when organisers were consistently generous with both varieties of attention throughout. But those were few and far between. The majority of organisers only paid attention to their more lively and popular campaigns (i.e. those least in need of attention), not bothering to try the proactive approach for the slow-starting ones.

To relate this problem to the DBAS framework, it may be that organisations expect too much before transitioning from the Broadcast stage to the Attract stage. By refraining from interaction with contributors until activity levels have crossed a certain threshold, organisers may be allowing potentially valuable campaigns to peter out needlessly.

Moreover, we found that attention paid to first-time contributors was especially effective at driving engagement, since newbies are highly receptive to organisational signals. Vigorous recruiting efforts aimed at newcomers in the Broadcast stage, therefore, should be accompanied by equally robust attention and responsiveness in the Attract stage.

Rejection – Launching a successful crowdsourcing campaign means rousing a lot of hopes that are destined to be disappointed. More than 90 percent of ideas from the crowd will not be used. Whether out of laziness or passive-aggression, most crowdsourcing organisers seem unwilling to squarely acknowledge this part of the process. Overall, we found that 88 percent of contributors ultimately received no notification on the fate of their submission.

Organisations should rethink their practices vis-à-vis rejections. In our studies, contributors who received one – even if it was terse and boilerplate – were far more likely to participate in future crowdsourcing campaigns managed by the same organisation. When organisers took the time to respond in language that stylistically resembled the contributor’s own communications, the likelihood of future engagement was even higher. We concluded that far from pushing people away, rejections actually bonded recipients even more tightly to the host organisation.

These surprising findings suggest that DBAS is perhaps best thought of as a cycle, with each misstep (and victory) carrying implications not only for the current campaign, but also all campaigns to come. Maybe crowdsourcing should be treated as a continual iterative churn, like the rapid innovation processes for which Silicon Valley tech firms are renowned.

Organisations’ crowdsourcing efforts could thereby be designed to protect a resource more valuable than any single idea or innovation – the loyalty of their best customers in the crowd.

Henning Piezunka is an Assistant Professor of Entrepreneurship and Family Enterprise at INSEAD.

Linus Dahlander is a Professor of Strategy at ESMT Berlin.

Lars Bo Jeppesen is a Professor at Copenhagen Business School.

This article was originally published by INSEAD Knowledge