Mobile Checkout Testing

Brief

Task/Challenge: As with every iterative design process, testing should be completed once work is published to a production environment. I authored a test evaluating various user tasks on mobile and published an A/B test to validate a recommended solution.

Toolbelt: User Testing, Competitive Research, A/B Testing

User Testing

“Ok, I don’t want to sign in. I don’t know if I really want to create an account. Is there no...oh”

9/10
users immediately concluded that they needed to sign in or create an account to checkout

Competitive Research

1/10
sites invited customers to create an account at this point in the funnel
4/10
sites skipped a checkout login page

A/B Testing

I set up an experiment in Monetate which hid the title bars and the "Create Account" button which were deemed to be noisy, as well as used jQuery to optimize the arrangment of content to better guide the user's eye down the desired path.

Control

Experiment

Results

$18k
incremental revenue
5.2%
lift in conversion
8.6%
lift in new customer conversion
-4.4%
lift in cart abandonment

The experiment ran for 2 weeks and achieved significance in all 4 metrics above. It was promoted to full-time and the changes have been added to the product backlog to become permanant changes.

Top