Edvard Osnaya← Back to portfolio
Case Study 02 · Research, Service Design & Usability Testing

Rebuilding trust in a score no one believed

An internal account-health and feature-adoption dashboard served three different teams, but no one had documented how they actually worked, and trust in the data had slipped. I led the research that mapped the real process, surfaced the pain points, and validated the beta features, producing a prioritized set of recommendations for the product team.

Enterprise SaaS / CRM platformAccount-health and feature-adoption dashboardInternal B2B tool + external betaMixed-methods research
RoleLead Designer · research & insights
TeamStrategy lead · UX researcher · Designer
Duration~8-week research program
Methods1:1 interviews · Service blueprint · Expert audit · Usability testing
16
1:1 discovery interviews across three roles
19
Usability-test participants · internal & external
15
Expert audit findings, severity-scored
3
New features validated with users
Context

One dashboard, three very different users

Inside one of the world's largest CRM platforms, an internal account-health and feature-adoption dashboard scores each account across three dimensions, product adoption, expertise, and technical health, instead of a single blended number. Customer Success Managers, Account Executives, and Renewal Managers use it to monitor adoption, prepare client conversations, and support renewals, while a customer-facing version helps clients track their own adoption and prove ROI.

It supports a lot of decisions, but no one had documented how the three roles actually worked day to day, and trust in the numbers had broken down. Before improving anything, the team needed a clear picture of the current process, its pain points, and where the data was losing credibility.

The Challenge

Data distrust had become the real product problem

The scores often didn't reflect how an account was actually performing, and the lack of visibility into how the model worked made it hard for users to have informed conversations with their customers.

Manual re-verification before every meeting

Users independently validated the data outside the platform before facing a client, because the numbers weren't always accurate.

Reports withheld to protect credibility

Account Executives had stopped sharing the auto-generated reports altogether, since system errors risked reflecting on their own professional competence.

A workflow scattered across five tools

To see unbundled product footprints and contract terms, users constantly context-switched between the scorecard, the CRM system of record, spreadsheets, and personal living documents.

Scores that moved for no visible reason

Back-end re-weighting could swing a score 20 points with zero customer action, leaving teams unable to explain a "win" or a "loss."

The biggest pain point is severe data distrust. I manually verify the data outside the platform before every client meeting, because a "0" could just be a tracking error.

Senior Customer Success Manager, discovery participant
Approach

Five research activities, one shared picture

The team had product documentation but no picture of how the three roles actually worked, and trust in the data had worn thin. I set three goals: understand the current process and its pain points, give the organization visibility into the end-to-end workflow, and validate the beta features so the team building them could iterate. Each activity answered a different question and fed a prioritized set of recommendations for the product team.

Layer 1 · Support-data analysis

Find the biggest problems first

I started by mining a full year of exported support-channel data to find the most frequent, highest-impact issues. That analysis set the priorities and shaped the interview script, so the 1:1 sessions could follow up directly on problems users were already reporting.

Layer 2 · Qualitative discovery

Learn how the three roles actually work

Product documentation existed, but the day-to-day process, challenges, and limitations of the people using the platform did not. I ran 16 in-depth 1:1 interviews (45-minute deep dives) across all three roles, and turned them into working personas built on real jobs, not job titles.

  • 9 Customer Success Managers, including lead, technical, and account-dedicated CSMs.
  • 4 Account Executives, core AEs and service specialists.
  • 3 Renewal Managers, senior and principal.
Layer 3 · Service design

Map the end-to-end process, for the first time

No documentation of the overall process existed. Mapped in parallel with the interviews and completed once they wrapped, an end-to-end service blueprint traced the customer lifecycle across all three roles, its actions, tools, and emotions, giving the organization a shared picture.

Layer 4 · Expert audit

Evaluate the interface against usability heuristics

A screen-by-screen heuristic evaluation produced 15 findings, each scored on a 0 to 4 severity scale, mapped to a usability heuristic, and paired with a concrete recommendation, ordered by impact.

Layer 5 · Beta feature testing

Validate the new features before a major release

In parallel, new beta features were being built for a major client event. Collaborating with the external UX/UI team designing them, I built the prototypes and test script, recruited internal and external users, and ran unmoderated remote sessions on a third-party usability-testing platform with 19 participants (8 internal, 11 external). The qualitative and quantitative report and its recommendations were presented back across several sessions.

The Blueprint

Three phases, three roles, one shared picture

The blueprint traced the full account lifecycle, showing at every step which tool each role reached for, what they felt, and where the platform quietly handed the work back to them.

Phase 1

Baseline adoption & health monitoring

Routine health checks and manual tracking to cover visibility gaps, plus hunting for upsell metrics in the utilization data.

Phase 2

Mid-cycle value realization & business reviews

Preparing customer-facing reviews, reconciling adoption data against contracted licenses, and rewriting AI-generated decks by hand.

Phase 3

Renewal strategy & expansion

Modeling pricing uplifts in spreadsheets, hunting shelfware to protect the negotiation, and opening the renewal conversation 90 days out.

The blueprint became the artifact everyone pointed at: product, engineering, and leadership finally looking at the same map instead of at each other.

Findings

Findings, scored and owned

The expert audit produced 15 findings on a 0 to 4 severity scale, and the research analysis used the same scale. Every issue below, from the audit or the interviews, carried a severity, a heuristic, a recommendation, and a stated impact, so prioritization stayed a conversation about evidence, not opinion.

2
5
4
4
2 catastrophic · imperative to fix 5 major 4 minor 4 cosmetic
Sev 0

Unexplained score fluctuations

Pain pointScores often didn't match the account's actual performance or state, and the platform gave no visibility into how a score was calculated, so teams couldn't explain wins or losses, and the scorecard lost credibility with clients.
RecommendationA "score insight" transparency drawer that separates customer-driven changes from system/algorithm updates, plus a proactive alert whenever the model changes.Visibility of system status
Sev 0

Unclear metric names

Pain pointMetric names were dense and undefined in-product, pushing users out of the platform to research terminology and fracturing the workflow.
RecommendationInline definitions on hover, in plain language, with the business value of each metric, keeping the user in the workflow.Recognition rather than recall
Sev 1

Bundled product data

Pain pointSpecific products did not appear as standalone line items. Grouped inside broader categories, they left renewal managers unable to tell whether a customer was actually using the capabilities in their contract.
RecommendationSurface every contracted product as its own line item, with a detail view that splits bundled contributions.Match with the real world
Sev 2

Metrics buried in a hierarchy tree

Pain pointFinding one specific metric meant clicking through nested subcategories and waiting on page loads, with no guarantee it lived where users expected.
RecommendationAdd metric search, plus "recently viewed" and pinned metrics.Flexibility & efficiency of use
Validation

Three new features, tested with real users

Ahead of a major client release, I validated the beta redesign with unmoderated remote sessions across internal and external users. The overall reception was positive: people grasped the core sections and welcomed more current, more actionable metrics. Testing focused on three new features.

Feature 1

AI-assisted actions

Resolved
Placement settled by testing

We tested where to place the control that launches an AI-assisted action in the table. The test settled it and exposed an expectation gap: people expected static, self-serve help, so we routed AI through a single, clearly-labeled entry point instead of repeating it per row.

Highlight
Feature 2

Recommendations section

Well received
A full new section

A full new section that recommends features to enable, tied to the business objectives each one supports. Easy to find and valued for showing the objectives a feature would impact, with clear notes on titles, terminology, and controls to refine.

Feature 3

Critical alerts

Standout
Banner + alerts tab

A prominent alert banner with a dedicated alerts tab. The strongest-received feature of the round, easy to spot and act on without losing context, with room to extend it to consumption alerts.

Overall, the new design landed well. The report separated what could ship as a baseline from the friction still worth refining, so the team building the features could keep iterating on evidence, not opinion.

Outcomes

What the work put on the table

Reframed the problem
The research reframed the core issue from data accuracy to explainability and trust, and its recommendations targeted that directly: a transparency drawer separating customer-driven from system-driven changes, proactive change alerts, and in-product metric definitions.
Validated a release
Three beta features were tested and refined before a major product release, with the winning variants and a documented list of the friction still to solve.
An end-to-end service blueprint
A service blueprint mapped the full customer lifecycle across all three roles, its actions, tools, and emotions, surfacing every workaround the platform was quietly forcing.
Severity-scored findings
The expert audit produced 15 findings, each with a severity, a heuristic, a recommendation, and an expected impact, ready to prioritize on evidence.
Reflection

What I took from it

As companies grow, the product and the experience around it can't stand still, they have to keep iterating alongside the organization. What this project reinforced for me is that understanding what the three roles actually needed, rather than assuming it, is what prevented the workarounds and the mistrust that had built up around the tool. Research at this scale is a small investment next to what it protects: a product people trust enough to use as intended, and the ROI it exists to prove in the first place.

ENES