SIM Calculator - A/B Test

Case Study: SIM Only Page Optimisation

Role
UX & User Research Specialist

Project Type
Conversion Rate Optimisation / UX Experiment

Problem Statement

SIM-only customers faced 24 plan options upfront, leading to choice overload, difficulty comparing features, and slower decision-making.

Goal

Enable customers to build their own SIM plan based on their needs, while still offering a curated selection of popular plans for quick and confident choice.

Documentation and Communication

Documentation
Keep detailed records of your design decisions, research findings, and user testing results.
Collaboration
Communicate effectively with cross-functional teams, including developers, product managers, and stakeholders, to ensure a shared understanding of the design goals.

Full Problem Statement

On the SIM Only page, multiple named tariffs were displayed in a single, long list.

This led to:

User confusion: Users struggled to understand the differences between plans.
Difficulty filtering: The flat list format did not allow users to narrow down plans based on their needs.
Overwhelming experience: Too many options presented at once created decision fatigue.

Hypothesis

We believed that introducing a plan calculator on the SIM Deals page would help users filter options according to their preferences.

Expected Outcome:

Users could quickly narrow down plans based on plan length, data amount, and plan type.
Increased add-to-basket rates and SIMO orders.

Success Criteria:

10% increase in add-to-basket rate
7.5% increase in SIMO orders
Results with 95% statistical significance

Evaluate, and acquire knowledge

Subsequently, the conceptual designs are transformed into functional interfaces, and are gradually introduced to a subset of the audience. These new designs are closely monitored over a three-week span to assess their impact on the core user journey, ensuring that there are no adverse effects.

Following the observation period, the data and insights collected are analysed. The findings are then presented in a stakeholder review session, where a recommendation is proposed.

The recommendation can take various forms, including either pausing and archiving the test, making iterative improvements, or integrating the design into the regular business-as-usual (BAU) process for continued development.

This decision is made based on the effectiveness and reception of the new design, as well as its alignment with the overall goals and strategies of the project.