How can I help you? (UX ROI traps)

In my previous post, I reported on recent hard data about the Return of Investment (ROI) of design and a long-term and shared perspective on user experience, and I showed, based on that data, that a proper UX process and culture can help companies grow.

Ok then. User experience pays. But what’s the catch?

Yep, there is a catch.

Here’s the kicker: to achieve these results, you need to do UX properly. And to do it properly, you need to plan the time, the budget, the people, and do it early. You need to iterate frequently and make it a shared responsibility of everyone in the project.

In my review, I found eight UX ROI traps that prevent user experience to bring actual benefits:

  • Expecting results with no budget and in no time. Needless to say, but let’s: if you don’t invest, you won’t get results. Lack of budget and planning often means having UX teams of one, with little or no time for research with users. One problem is that quick development cycles often value quantity over quality. Under pressure, it is easy to push aside user research. But proper user research is what gives your product a competitive advantage (see my previous posts on this, What UX is not, The process of UX and The ROI of UX). UX is not magic and cannot be based solely on the designer’s experience: s/he may know usability best practices, cognitive biases and UI patterns, but insufficient user research will make you rush into development with design assumptions at best, not design findings. Veeery risky.
  • Interviewing 5 users is just not enough! The sample size of 5 largely comes from an old study by Nielsen and Landauer in 1993 (and here’s a more recent divulgative article by Nielsen), where they found that you can find the majority of usability problems in a product by testing it with 5 users. After the fifth user, you’re basicaly wasting time. Well, since then the number 5 has become the rule of thumb for every type of user testing, no matter the scope, the type of test, or the variety of user groups. But let’s consider that when it was proposed, in 1993, there was a strong need to show engineers that even cheap usability testing could bring significant results. Now, products are much more complex and users are much more demanding. To estimate your sample size, you should ask yourself:

    How many user groups will use my product? Test with at least 5 users per user group, or as close as you can get.

    What type of test is this? Is it a usability study or a field study? Usability studies can be done with as few as 5-8 users. Field studies require more users, a rough estimate being around 15-30.

    Are you testing a prototype or the final product? If you’re testing a prototype 8 users could be enough, if you’re testing the final product it’s better to go with at least 15 users.

    Bottom line: if your users are not enough, you risk founding your design on assumptions and making poor design decisions. That being said: 5 users is better than no users at all!

You cannot dig a hole in a different place by digging the same hole deeper.
— Edward de Bono

  • Skipping divergent thinking due to time constraints (and getting stuck with the straighforward solution). Or, in the words of Edward De Bono, the father of lateral thinking: “You cannot dig a hole in a different place by digging the same hole deeper”. Ideation matters: thinking of different ways to look at the problem and generating alternative design solutions helps to understand the design space and avoid mediocre results.
  • Thinking that you are your user. You’re not. Thinking that something will work because you find it useful, easy and valuable is an insidious trap. To some extent, we’re all victims of confirmation bias. Loudest opinion. Best guesses. And groupthink. Base your design of thick data, not on opinions.
  • Work in silos (and not talking). User experience is a shared responsibility, and requires coordination and collaboration (remember the McKinsey Design Index in my previous post?). UX designers may be accountable for the design process from user research to interaction design, but many other actors provide significant elements for a good (or bad) user experience: developers, engineers, sales, marketing, all contribute to the puzzle. Ignoring this will likely result in cripple products. If there’s a missing piece, it stands out. You can design the most wonderful digital interface, but if the architecture is not solid, if it keeps crashing, if it has security bugs, if customer support does not provide a great service, the whole customer experience will be faulty. Achieving this kind of awareness often implies a cultural change that may require months or even years of baby steps to pervade an entire organization. So start small, from individual projects: set a process, commit to it, test it, iterate, share. Then, start again. Monitor the process for the whole time, with retrospectives and workshops like Lightning Decision Jam, which can point you to the right direction to efficiency.
    PS: Did you know there’s a Miro template for Lightning Decision Jam sessions by AJ&Smart?
Working in silos will likely result in cripple products. If there’s a missing piece, it stands out (Picture by Ann H from Pexels).
  • Not testing/iterating. Doing the user research, then the design only once won’t get you very far. Let’s bust a myth here… Designers don’t get it right the first time! And believe it or not, designers do make mistakes! Everyone makes mistakes, and designers are not immune. The clever thing to do is to find those mistakes and correct them. You need to iterate. And you cannot do it alone. Test with users in small iteration cycles, involve developers, project managers, if possible even clients and relevant stakeholders to find together the best solutions to usability and UX problems.
  • Not using the UX deliverables. So, you did the most thorough user research, found needs, pain points and desiderata, analysed the data, identified key insights and produced personas, customer journey maps, user stories, and a nice taxonomy from participatory card sorting. Then what? Nobody looks at these deliverables and few cares. And the insights from hard data will not be included in the design to an extent that matters. People will keep making decisions based on assumptions and opinions, not on data. If you have the data, use it. Share it. Present it. Let everyone involved know about the value of data.
  • Not measuring KPIs. With KPIs (Key Performance Indicators) not only can you measure the benefits of a UX process, you can also find room for improvement, you have a way to communicate effectively within the team, with other departments and with management. You can move from abstract and vague concepts to concrete talk, and to show the business impact of design.

Of course, in a perfect world we would avoid all these traps, follow a thorough design process in a shared culture of user-centered design, and create beautiful experiences with our products. In the real world, we do what we can. But this doesn’t mean we should stop trying and do our best.

On that note, here’s an interesting talk by Kerry Bodine, one of the authors of “Outside In: The Power of Putting Customers at the Center of Your Business” (Manning, Bodine, & Bernoff, 2012), where she explains the business value of customer experience (here’s a video presentation of the book).

Kerry Bodine taking about the importance of putting users at the center of the design of products, connecting to the users’ hands (making the product effective), head (making the product easy to use), and heart (being emotionally relevant).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s