User Testing vs. A/B Testing

So you have a page, or a system, or a form or an app or anything, and you know you want to make it ‘better’. And the question is… Should we use User Testing or A/B Testing? TL;DR: Both can be valuable, and you can do both. So long as you have an underlying indicator of the ‘better’ that you can track over the long-term. A/B Testing A/B testing is good because: It shows you ‘for real’, at scale, what actually prompts your users to do the things you most want them to do It can further polish processes that have been through thorough user testing; to a degree that’s just not possible with a small group of people It can self-optimize content in real-time to maximize the impact of short spikes of campaign activity A/B testing is bad because: … Continue reading »

Weeknotes 5 – Webmaker Workweek

As I’m halfway into the following week I’m writing these notes quickly rather than losing them completely. I apologize in advance Week 5 was spent in Toronto with the Webmaker team and it will be hard for a quick write-up to do this week justice. I got to hack and hang-out with about half of the total Mozilla Foundation staff, which is hugely valuable four weeks into a job where you mostly work remotely. IRC handles turned into real people, and the people turned out to be very special. So first, thanks to this amazing team for welcoming me so kindly. I think we crammed a year’s worth of social activity into a week’s worth of evenings and across the whole week, I almost got a whole night’s worth of sleep. There’s a test that goes something along the lines … Continue reading »

Something I wrote for Engaging Networks

A few weeks ago I received a marketing email from the Engaging Networks team quoting some stats about the possible improvements to website conversion rates that can be achieved with A/B testing. I was caught off guard (but pretty chuffed) when I realised I was being quoted my own case study from a presentation I had given a couple of years earlier. I sent a quick reply to the email and was delighted to find it was sent from a real address with a real person at the other end reading the replies (@Rachel_shares). This turned into a nice discussion about conversion rate testing, and somehow I agreed to write a guest blog post. Which, with some helpful editing from Rachel has now been posted on the Engaging Networks blog. I thought I should share the link with all two … Continue reading »

Five Second Tests

Link: Five Second Tests A nice way to find out people’s actual first impression of your web page or design. It’s a good idea to to register for a free account and complete some tests for others, even if it’s just to help you think about your own page designs.