I'm Adam. Nice to meet you.
My Twitter bio:
Metrics Lead at Mozilla Foundation. Previously at WWF. Test, tinker, and question the answers.
If I find a moment, I’ll write about many of the fun and inspiring things I saw at Mozfest this weekend, but this post is about a single session I had the pleasure of hosting alongside Andrew, Doug and Simon; Learning Analytics for Good in the Age of Big Data. We had an hour, no idea if anyone else would be interested, or what angle people would come to the session from. And given that, I think it worked out pretty well. We had about 20 participants, and broke into four groups to talk about Learning Analytics from roughly 3 starting points (though all the discussions overlapped): Practical solutions to measuring learning as it happens online The ethical complications of tracking (even when you want to optimise for something positive – e.g. Learning) The research opportunities for publishing and connecting … Continue reading
I’m back at the screen after a week of paternity leave, and I’ll be working part-time for next two weeks while we settle in to the new family routine at home. In the meantime, I wanted to mention a Mozilla contributor analysis project in case people would like to get involved. We have a wiki page now, which means it’s a real thing. And here are some words my sleep-deprived brain prepared for you earlier today: — The goal and scope of the work: Explore existing contribution datasets to look for possible insights and metrics that would be useful to monitor on an ongoing basis, before the co-incident workweek in Portland at the beginning of December. We will: Stress-test our current capacity to use existing contribution data Look for actionable insights to support Mozilla-wide community building efforts Run ad-hoc analysis … Continue reading
Removing the second sentence increases conversion rate (hypothesis = simplicity is good). The button text ‘Go!’ increased the conversion rate. Both variations on the headline increased conversion rate, but ‘Welcome to Webmaker’ performed the best. We should remove the bullet points on this landing page. The log-in option is useful on the page, even for a cold audience who we assume do not have accounts already. Repeating the ask ‘Sign-up for Webmaker’ at the end of the copy, even when it duplicates the heading immediately above, is useful. Even at the expense of making the copy longer. The button text ‘Create an account’ works better than ‘Sign up for Webmaker’ even when the headline and CTA in the copy are ‘Sign up for Webmaker’. These two headlines are equivalent. In the absence of other data we should keep the version … Continue reading
Here are a couple of notes about ‘Hack the snippet‘ that I wanted to make sure got documented. It significantly changed peoples’ predisposition to Webmaker before they arrived on the site Its ‘post-interaction’ click-through-rate was equivalent to most one-click snippets Behind these observations, something special was happening in ‘Hack the snippet’. I can’t tell you exactly what it was that had the end-effect, but it’s worth remembering the effect. 1. It ‘warmed people up’ to Webmaker The ‘Hack the snippet’ snippet was shown to the same audience (Firefox users) as eight other snippet variations we ran during the campaign had the same % of users click through to the landing page had the same on-site experience on webmaker.org as all the other snippet variations we tested (the same landing page, sign-up ask etc) But when people who had interacted with … Continue reading
This post is an attempt to capture some of the things we’ve learned from a few busy and exciting weeks working on the Webmaker new user funnel. I will forget some things, there will be other stories to tell, and this will be biased towards my recurring message of “yay metrics”. How did this happen? As Dave pointed out in a recent email to Webmaker Dev list, “That’s a comma, not a decimal.” What happened to increase new user sign-ups by 1,024% compared the previous month? Is there one weird trick to…? No. Sorry, I know you’d like an easy answer… This growth is the result of a month of focused work and many many incremental improvements to the first-run experience for visitors arriving on webmaker.org from the promotion we’ve seen on the Firefox snippet. I’ll try to recount some … Continue reading
TL;DR: Check out this graph! Ever wondered how many Mozfest Volunteers also host events for Webmaker? Or how many code contributors have a Webmaker contributor badge? Now you can find out… The reason the MoFo Contributor dashboard we’re working from at the moment is called our interim dashboard is because it’s combining numbers from multiple data sources, but the number of contributors is not de-duped across systems. So if you’re counted as a contributor because you host an event for Webmaker, you will be double counted if you also file bugs in Bugzilla. And until now, we haven’t known what those overlaps look like. This interim solution wasn’t perfect, but it’s given us something to work with while we’re building out Baloo and the cross-org areweamillionyet.org (and by ‘we’, the vast credit for Baloo is due to our hard working … Continue reading
TL;DR Our MoFo dashboards now have trendlines based on known activity to date The recent uptick in activity is partly new contributors, and partly new recognition of existing contributors (all of which is good, but some of which is misleading for the trendline in the short term) Below is a rambling analogy for thinking about our contributor goals and how we answer the question ‘are we on track for 2014?’ + if you haven’t seen it, OpenMatt has crisply summarized a tonne of the data and insights that we’ve unpicked during Maker Party Stacking Logs I was stacking logs over the weekend, and wondering if I had enough for winter, when it struck me that this might be a useful analogy for a post I was planning to write. So bear with me, I hope this works… To be clear, … Continue reading
We’re a little over halfway through the year now, and our dashboard is now good enough to tell us how we’re doing. TL;DR: The existing trend lines won’t get us to our 2014 goals but knowing this is helpful and getting there is possible Ask less: How do we count our contributors? Ask more: What are we doing to grow the contributor community? And, are we on track? Changing the question Our dashboard now needs to move from being a project to being a tool that helps us do better. After all, Mozilla’s unique strength is that we’re a community of contributors and this dashboard, and the 2014 contributor goal, exist to help us focus our workflows, decisions and investments in ways that empower the community. Not just for the fun of counting things. The first half of the year … Continue reading
We just started using our first webmaker.org landing page, and I thought I’d write about why this is so important and how it’s working out so far. Who’s getting involved? Every day people visit the webmaker.org website. They come from many places, for many reasons. Sometimes they know about Webmaker, but most of the time it’s new to them. Some of those people take an action; they sign-up to find out more, to make something with our tools, or even to throw a Maker Party. But, most of the people who visit webmaker.org don’t. The percentage of people who do take action is our conversion rate. Our conversion rate is an important number that can help us to be more effective. And being more effective is key to winning. If you’re new to thinking about our conversion rate, it can … Continue reading
tl;dr: We’re wrapping up work on the MoFo Interim Dashboard The only other data source we’ll add is the badge counts for webmaker mentors & hive community members This is still our MoFo working document for the time being Then, we’ll switch our development efforts into integrating with Project Baloo Baloo is where we will de-dupe contributors across teams/tools etc. areweamillionyet.org will become our working document in time (‘time’ is TBC) Switching to Plan A The MoFo contributor dashboard we’ve been working with this year is our *interim* counting solution, and just as we’re “completing” it we’re now in a position to switch from an interim solution to a fully integrated system which is properly integrated with MoCo. This is pretty good timing, but it’s a change in scope for our immediate work so is worth a status update. Within … Continue reading