This article reports the results of a survey, in which 436 UX practitioners completed (or attempted to complete) an online survey in October 2019.

The discovery phase is an important part of the UX-design process, so we aimed to understand how companies perform discoveries on real-world design projects and the barriers to carrying out discoveries effectively.

A reminder from our full article on discovery in UX: A discovery is a preliminary phase in the UX-design process that involves researching the problem space, framing the problems to be solved, and gathering enough evidence and initial direction on what to do next.

Survey Findings

Most UX Practitioners Carry Out Discoveries for Some Projects

75% of the respondents claimed that their organization carries out discoveries. Out of those, only 20% said that their organization performs discoveries for every project, while the majority (60%) claimed that their organization carries out roughly 1 discovery phase for every 2–4 projects.

Many Organizations Forgo Discovery Due to Lack of Time

The top reason for not carrying out discoveries at all in an organization was time pressure to deliver, followed by no management buy-in. Only 7% of the 69 respondents whose organizations did not carry out discoveries believed lack of resources was the biggest factor.

“[The] focus is on delivery — they want to deliver ‘something’, and then fix later.”

It’s worth noting that the desire to deliver quickly may also affect UX professionals’ work in other stages of the design process, particularly in carrying out usability testing or having UX insights incorporated into product development. Rushing into implementation with too little time for UX activities is a sign of an organization’s low UX maturity.

A bar chart shows the top reason given by 75 UX practitioners for why their organization does not carry out discovery phases in any of their projects. 35% of the respondents reported it was due to time pressure to deliver. 28% reported lack of management buy-in being the top reason. 23% reported the top reason being a lack of knowledge in the organization on how to carry them out. Only 7% of respondents said the top reason was a lack of resource. 8% said it was another reason not listed.
75 respondents shared the one top reason why their organization doesn’t perform discoveries at all as part of its design process. The graph indicates the distribution of their answers.

Of course, in reality, several factors dissuade organizations from implementing discoveries, and our respondents explained that it was often a combination of both management buy-in and time pressure.

“In addition to time pressure, there is little CEO buy-in. He tends to feel that we already know the problem that needs to be solved.”

“I think there's also a lack of understanding of their value — the company considers itself very innovative, and there's a race to come up with the next amazing idea.”

Interestingly, respondents that work for organizations that do run some discovery phases in their design projects reported (in a free-text field) the same reasons for why certain projects did not get a discovery phase. Both these findings suggest that lack of buy-in and time are top challenges for running discoveries.

UX Practitioners Who Conduct Discoveries Are More Likely to Report that Their Projects Are Successful

Respondents were asked about the last project they worked on at their organization. Of those who did not carry out a discovery, 52% of respondents said that their last project was successful — barely better than chance! In contrast, 83% of respondents who had carried out a discovery in their last project said that the project was successful. A chi-square test revealed this difference to be highly significant (p<.00001).

Although we can’t be certain if these projects were actually successful and we don’t know whether it was the discovery phase which contributed to the success of the project rather than correlated factors, this result does lend credence to the widely held belief that discoveries lead to better outcomes, because they frame the problem to be solved, set teams in the right direction, and provide them with relevant knowledge to design successful solutions. Further evidence that this is the case was provided when respondents rated their agreement to the following statement: Our discoveries positively impact the products and services we build. 84% of 240 practitioners with experience of carrying out discovery phases agreed with the statement, and only 5% disagreed.

A bar chart shows the percentage of respondents who reported their last project to be successful. Of those projects that had a discovery, 83% were reported to have been successful, compared to 52% of projects that did not have a discovery. Only 4% of projects with a discovery were unsuccessful, compared to 16% of projects that did not have a discovery.
The bar chart shows the percentage of respondents who reported their last project to be successful. People who worked on a project with a discovery phase were more likely to report success than those whose projects did not have a discovery phase. Error bars indicate the 95% confidence interval.

There are two ways of interpreting this chart:

  • Performing a discovery cuts the risk of failure by 75% (4 vs. 16 failures).
  • Performing a discovery increases the chance of success by 59% (83 vs. 52 successes).

Of course, these two ways of stating the results are two sides of the same coin, but it is remarkable that the most striking impact of discoveries comes from risk reduction. This is actually true of most UX work: doing good UX will often increase product success, but it will almost always insure you against true disasters.

The Average Team Size Is 4

We asked respondents to tell us who was on the team carrying out the discovery. The average number of individuals involved full time on a discovery project is 4 (SD=3.5), with a range from 1 to 36 (which is far too many). The most common roles present on a discovery project were UX or product designer (74%), researcher (60%), and product or project manager (52%). Only 36% of discoveries had a product owner.

A bar chart shows the percentage of 248 respondents who indicated that the following roles were present on their last discovery project. 74% said a designer was present. 60% said a researcher was present. A product or project manager was present in 52% of respondents' discovery phases. A product owner was present on 36% of respondents' discoveries. Between 20-30% of projects had a business analyst (28%), a developer (27%), a subject matter expert (23%), and a visual or graphic designer (20%). The most uncommon roles present on discoveries include a content designer (17%), a technical architect (15%), a service designer (13%) and a scrum master (9%).
The bar chart shows the percentage of respondents who indicated that these roles were present on their discovery team.

Most Discoveries Are Carried Out in Under 2 Weeks

We asked respondents to tell us how long the last discovery they ran was. Obviously, all discoveries are different, but we wanted to know roughly how long the average discovery phase is, across all projects.

A bar chart shows the percentage of 242 respondents reporting the length of their last discovery. 29% reported their discovery was under 1 week. Another 29% said their discovery was longer than 1 week, but shorter than 2 weeks. 10% of respondents reported their discovery to be between 3 and 4 weeks. 6% of respondents reported their last discovery was between 5-6 weeks. A higher percentage (16%) claimed their discovery was between 7 and 8 weeks, and 11% claimed their discovery ran for more than 8 weeks.
The chart shows distribution of discovery lengths, as given by 242 UX practitioners. Each bar indicates the percentage of practitioners reporting the corresponding length for their last discovery. A majority of respondents (58%) reported that their last discovery lasted at most 2 weeks.

I would have expected the data to form more of a bell curve, with few participants running very short or very long discoveries, but instead the chart above is somewhat bimodal. The majority (58%) of discoveries undertaken by respondents were on the short side — at most two weeks. Unless users are easy to access and the problem space is relatively small, two weeks is too short. Just under a third perform longer discoveries (over 7 weeks).

There are many possible reasons why the majority of discoveries are short. One may be related to budget and time constraints, which result in time-limited discoveries. Another reasons might be the popularity of Google Ventures’ Design Sprint, as well as the adoption of continuous product discovery, which means some organizations favor leaner initial discovery phases, while carrying out more discovery-style research throughout the design process.

The problem with slimming down the discovery phase or replacing it with a design sprint is that some teams may miss out on a time of crucial learning, beginning the design process with solution ideas that might not be based on real problems. Survey respondents commented on the brevity of discoveries in their organization:

“In my opinion, the time spent on discoveries in my company is not enough and sometimes discoveries are carried out not to know the problem better, but to confirm [the] initial idea.”

“The UX team believes discovery work can be continuous with 5 or less participants and always presenting a solution first without researching the problem space first, just going off anecdotal evidence on problems. 2–3 people's response to a proposed solution that doesn't address the problems they've actually expressed doesn't seem like best practice.”

Most Common Activities

In the discovery phase, it’s possible to perform many different activities: stakeholder interviews and workshops, analyzing analytics data, field studies. Respondents were asked which activities they had performed in their last discovery. When carrying out exploratory research, it’s best if data is triangulated from numerous sources and through different methods, to ensure that teams are getting a full picture of the problem.

A bar chart shows the percentage of 249 respondents that reported doing 13 different activities during their last discovery phase. The most popular activities were user interviews (62%) and stakeholder interviews (60%). Between 40-50% of respondents carried out competitive analysis (49%), analyzing analytics (49%) and usability testing (42%). Around one-third of respondents carried out surveys (37%), co-design workshops (34%) and stakeholder workshops (33%). Less than one-third of respondents carried out field studies (28%), card sorts (24%), design sprints (24%), focus groups (17%) and diary studies (6%).
The bar chart shows the percentage of respondents who reported performing each of 13 discovery-phase activities.

Across all discoveries, the average number of activities performed was 4.6 (SD=2.5). The most common activities carried out were user interviews and stakeholder interviews. Both methods are quick, which would explain their popularity, given that many discoveries are run in less than 2 weeks. The short duration of discoveries might also explain why diary studies are the most unpopular activity.

Biggest Challenges in Carrying Out Discoveries

Many discoveries are far from perfect and are hampered by real-world constraints. Respondents were asked what their biggest challenges were in conducting discoveries effectively. 215 responses were provided via a free-text field; these responses were coded and analyzed. There were many different challenges, so we simply report on the most frequent challenges mentioned by respondents.

Lack of Time and Budget

The most frequent blocker mentioned was limited time (84 comments). Budget (27 comments) was also a factor; note that budget also affects allocated time. Even if the organization plans to do a discovery, the discovery itself is often the first thing to be reduced or removed.

“The intent is there to allow time for discovery but then deadlines and bottom-line revenue is pushed by old mindsets and sadly everything falls back 50 years to system development, and we try our best to [do] some form of discovery.”

In some cases, UX is simply brought in too late, which means UX practitioners have less time to carry out essential discovery work.

“It isn't often built into the project process, or teams come to us too late to do anything. Even if we discover the solution doesn't address a problem, too much development work has been put into it to stop…”

In other cases, respondents spoke about not having enough budget to perform longer discoveries, because time is money, especially for agencies pitching work to clients.

“Demonstrating the value of comprehensive discoveries to [the] public sector. This sector is very accustomed to penny pinching and it can seem like a ‘luxury item’ when they already believe they have identified the problem(s).”

Budget constraints can also affect things like how big the team is and whether there is enough money for incentives or recruiting for user research.

Lack of Buy-in and Hard to Prove Value

Another big theme was lack of buy-in (61 comments). This means UX practitioners are constantly trying to explain the value of the discovery phase, and are sometimes not successful in convincing stakeholders or clients to do a proper discovery due to “legacy ways of thinking,” beliefs that product managers already know their users, that solutions are obvious, or that shipping a Minimum Viable Product (MVP) will provide all required learning.

“Again, selling in a discovery-based project is tough. Whilst we advocate for it and show benefits, some projects still believe [that] shipping [an] MVP will give them the learning they require. Discovery can provide good direction, reducing risk and increasing confidence of the product.”

Many respondents spoke about stakeholders and other team members not understanding what the discovery is. Some specifically mentioned the difficulty in quantifying the ROI of a discovery, which is often needed to gain buy-in from those holding the purse strings.

The Right People Aren’t Always Involved

16 respondents mentioned challenges related to getting the right people on the team or to lead the team.

“Even though every project starts with discovery, it is still a largely inconsistent process and it is not currently led by product managers or designers. Instead, it is led by project managers, engineers, or account managers. We're working to change this, but product managers and designers are still largely left out of discoveries leading to pain later on in the process.”

Some respondents were a team of one, and, thus, responsible for doing discovery alone. Others mentioned that the people with the right skills weren’t involved, which made running the discovery difficult. For example, some teams lacked a person able to competently carry out user research.

“Putting together a multidisciplinary team is a tad difficult when we have limited staff with the right skill sets.”

Lastly, other respondents claimed that people with a lack of understanding were also on the team, and it was hard to collaborate with them or prevent them from jumping to solutions and succumbing to their own biases.

Access to Users

30 respondents mentioned getting access to users was difficult in their discoveries. Several explained that they worked for B2B organizations and were denied access to users or the process to recruit them was very difficult. This problem was exacerbated by lack of time to recruit specialized users. This problem is not unique to the discovery phase, and we hear about it often from our attendees at our UX conferences.

Survey Limitations

We invited UX practitioners to complete the survey through LinkedIn, Twitter, our Alertbox newsletter, and our website. Since we recruited a convenience sample, we cannot claim that it is representative. It’s likely that in our sample, UX practitioners with experience of carrying out discoveries are overrepresented. We tried to minimize this effect by calling for all UX practitioners to complete the survey, regardless of whether their organization does or doesn’t run discovery phases.

Conclusion

While many UX practitioners have succeeded in implementing discovery phases in their organization, most discoveries are still too short, underresourced, and don’t have the full backing of the organization. This is often the case when clients and stakeholders don’t understand their value. This attitude leads to cost-cutting and trimming of what are deemed unnecessary activities. As a result, too many design projects timidly explore already familiar corners of the design space, missing out on building features that satisfy previously unmet user needs. The good news for those who do perform discoveries is that they possess a ‘secret weapon’ that will help them establish a sustained competitive advantage.

As a profession, we’ve done well to find our place in the development process, convincing others that we should test with users. But we need to become better at educating our organization as to why it makes business sense to spend time upfront in discovery, and why getting UX practitioners involved early is important. Imagine how much more successful projects would be if we could make that happen!

Learn More: Discoveries: Building the Right Thing, a full-day course at the UX Conference.