Usability Testing with Users – Methods, Benefits, and When to Use Them

By Philip Burgess – UX Research Leader

Usability testing is one of the most powerful tools in a UX researcher’s toolkit. It helps teams uncover how real users interact with a product, where they struggle, and how to improve the experience. While “usability testing” is often used as a catch-all term, there are actually several distinct types, each suited for different goals and phases of the design process.

Below, we’ll explore the main usability testing approaches you can use, their benefits, and when they work best.

1. Usability Testing

What it is: The classic form of usability testing involves observing users as they attempt to complete key tasks in your product or prototype. The goal is to identify usability issues, measure task success, and gather feedback.

When to use it:

• Early in design to identify pain points

• Before a launch to ensure key tasks are achievable

• Post-launch to refine existing features

Why it works: It focuses on actual user behavior rather than assumptions, giving you concrete evidence of what works and what doesn’t.

2. Benchmark Testing

What it is: Benchmark testing sets a performance baseline for your product by measuring metrics like task completion rates, error rates, and time on task. This can be repeated over time to track improvements or regressions.

When to use it:

• Before a redesign to measure the “before” state

• After updates to validate improvements

• As part of an ongoing usability program to measure progress

Why it works: It transforms usability into measurable, trackable data, making it easier to prove ROI for UX efforts.

3. Competitive Usability Testing

What it is: This method compares your product’s usability against competitors’ products or industry benchmarks. Users perform similar tasks in each product so you can see where yours stands out—or falls behind.

When to use it:

• Before launch to identify competitive advantages or gaps

• To guide redesigns with insights from market leaders

• For stakeholder alignment on where your product fits in the market

Why it works: It gives direct, actionable comparisons rather than vague market opinions.

4. Summative Usability Testing

What it is: Summative testing evaluates the overall effectiveness, efficiency, and satisfaction of a product—often near the end of a design cycle or after launch. Unlike formative testing (which is about exploring and iterating), summative testing is about validating readiness.

When to use it:

• Final validation before a major release

• To ensure compliance with usability standards

• When stakeholders need concrete evidence of usability

Why it works: It answers the question, “Are we ready to release this?”

5. Remote Evaluation

What it is: Users complete tasks remotely—either moderated in real-time or unmoderated on their own—without needing to be in the same physical location as the researcher.

When to use it:

• When your users are geographically dispersed

• To scale testing quickly with larger participant groups

• When budgets or time make in-person testing impractical

Why it works: It captures real-world usage scenarios and allows for faster recruitment and execution.

6. Think Aloud Testing

What it is: Participants verbalize their thoughts, feelings, and decisions as they navigate your product. This provides deep insight into their mental models and reasoning.

When to use it:

• Early in design to understand user thought processes

• To uncover why users make errors or struggle with navigation

• To explore expectations and assumptions

Why it works: It reveals the why behind user actions, not just the what.

7. Wizard of Oz

What it is: In Wizard of Oz testing, users interact with what they believe is a functioning system, but in reality, a human is simulating some or all of the system’s responses. This is often used to test concepts before full development.

When to use it:

• To explore new concepts without building complex back-end systems

• To test AI, chatbots, or other tech where development is expensive

• To refine interactions before investing in coding

Why it works: It lets you validate ideas and flows before committing resources.

Final Thoughts

Usability testing isn’t one-size-fits-all. Each method offers unique strengths, and the best approach often depends on your project’s goals, stage, and resources. By combining these methods strategically, you can uncover richer insights, validate decisions, and build products that truly work for your users.