I'm Adam. Nice to meet you.
My Twitter bio:
Metrics Lead at Mozilla Foundation. Previously at WWF. Test, tinker, and question the answers.
If I find a moment, I’ll write about many of the fun and inspiring things I saw at Mozfest this weekend, but this post is about a single session I had the pleasure of hosting alongside Andrew, Doug and Simon; Learning Analytics for Good in the Age of Big Data. We had an hour, no idea if anyone else would be interested, or what angle people would come to the session from. And given that, I think it worked out pretty well. We had about 20 participants, and broke into four groups to talk about Learning Analytics from roughly 3 starting points (though all the discussions overlapped): Practical solutions to measuring learning as it happens online The ethical complications of tracking (even when you want to optimise for something positive – e.g. Learning) The research opportunities for publishing and connecting … Continue reading
I’m back at the screen after a week of paternity leave, and I’ll be working part-time for next two weeks while we settle in to the new family routine at home. In the meantime, I wanted to mention a Mozilla contributor analysis project in case people would like to get involved. We have a wiki page now, which means it’s a real thing. And here are some words my sleep-deprived brain prepared for you earlier today: — The goal and scope of the work: Explore existing contribution datasets to look for possible insights and metrics that would be useful to monitor on an ongoing basis, before the co-incident workweek in Portland at the beginning of December. We will: Stress-test our current capacity to use existing contribution data Look for actionable insights to support Mozilla-wide community building efforts Run ad-hoc analysis … Continue reading
This post is an attempt to capture some of the things we’ve learned from a few busy and exciting weeks working on the Webmaker new user funnel. I will forget some things, there will be other stories to tell, and this will be biased towards my recurring message of “yay metrics”. How did this happen? As Dave pointed out in a recent email to Webmaker Dev list, “That’s a comma, not a decimal.” What happened to increase new user sign-ups by 1,024% compared the previous month? Is there one weird trick to…? No. Sorry, I know you’d like an easy answer… This growth is the result of a month of focused work and many many incremental improvements to the first-run experience for visitors arriving on webmaker.org from the promotion we’ve seen on the Firefox snippet. I’ll try to recount some … Continue reading
TL;DR: Check out this graph! Ever wondered how many Mozfest Volunteers also host events for Webmaker? Or how many code contributors have a Webmaker contributor badge? Now you can find out… The reason the MoFo Contributor dashboard we’re working from at the moment is called our interim dashboard is because it’s combining numbers from multiple data sources, but the number of contributors is not de-duped across systems. So if you’re counted as a contributor because you host an event for Webmaker, you will be double counted if you also file bugs in Bugzilla. And until now, we haven’t known what those overlaps look like. This interim solution wasn’t perfect, but it’s given us something to work with while we’re building out Baloo and the cross-org areweamillionyet.org (and by ‘we’, the vast credit for Baloo is due to our hard working … Continue reading
TL;DR Our MoFo dashboards now have trendlines based on known activity to date The recent uptick in activity is partly new contributors, and partly new recognition of existing contributors (all of which is good, but some of which is misleading for the trendline in the short term) Below is a rambling analogy for thinking about our contributor goals and how we answer the question ‘are we on track for 2014?’ + if you haven’t seen it, OpenMatt has crisply summarized a tonne of the data and insights that we’ve unpicked during Maker Party Stacking Logs I was stacking logs over the weekend, and wondering if I had enough for winter, when it struck me that this might be a useful analogy for a post I was planning to write. So bear with me, I hope this works… To be clear, … Continue reading
We’re a little over halfway through the year now, and our dashboard is now good enough to tell us how we’re doing. TL;DR: The existing trend lines won’t get us to our 2014 goals but knowing this is helpful and getting there is possible Ask less: How do we count our contributors? Ask more: What are we doing to grow the contributor community? And, are we on track? Changing the question Our dashboard now needs to move from being a project to being a tool that helps us do better. After all, Mozilla’s unique strength is that we’re a community of contributors and this dashboard, and the 2014 contributor goal, exist to help us focus our workflows, decisions and investments in ways that empower the community. Not just for the fun of counting things. The first half of the year … Continue reading
We just started using our first webmaker.org landing page, and I thought I’d write about why this is so important and how it’s working out so far. Who’s getting involved? Every day people visit the webmaker.org website. They come from many places, for many reasons. Sometimes they know about Webmaker, but most of the time it’s new to them. Some of those people take an action; they sign-up to find out more, to make something with our tools, or even to throw a Maker Party. But, most of the people who visit webmaker.org don’t. The percentage of people who do take action is our conversion rate. Our conversion rate is an important number that can help us to be more effective. And being more effective is key to winning. If you’re new to thinking about our conversion rate, it can … Continue reading
For our almost complete MoFo Interim Dashboard, I’m planning to use an issue tracker parsing tool called Bicho to work out how many people are involved in the Webmaker project in Bugzilla. Bicho is part of a suite of tools called Metrics Grimoire which I’ll explore in more detail in near future. When combined with vizGrimoire, you can generate interesting things like this which are very closely related to (but not exactly solving the same challenge) as our own contribution tracking efforts. I recently installed a local copy of Bicho, and ran this against some products on Bugzilla to test it out. It generates a nicely structured relational database including the things I want to count and feed into our contributor numbers. This morning I got this running on Heroku, which means it can run periodically and update a hosted … Continue reading
This is an idea for Webmaker teacher dashboards, and some thoughts on metadata related to learning analytics This post stems from a few conversations around metrics for Webmaker and learning analytics and it proposes some potential product features which need to be challenged and considered. I’m sharing the idea here as it’s easy to pass this around, but this is very much just an idea right now. For context, I’m approaching this from a metrics perspective, but I’m trying to solve the data gathering challenge by adding value for our users rather than asking them to do any extra work. These are the kind of questions I want us to be able to answer and that can inform future decision making in a positive way… How many people using Webmaker tools are mentors, students, or others? Do mentors teach many … Continue reading
As I’m halfway into the following week I’m writing these notes quickly rather than losing them completely. I apologize in advance Week 5 was spent in Toronto with the Webmaker team and it will be hard for a quick write-up to do this week justice. I got to hack and hang-out with about half of the total Mozilla Foundation staff, which is hugely valuable four weeks into a job where you mostly work remotely. IRC handles turned into real people, and the people turned out to be very special. So first, thanks to this amazing team for welcoming me so kindly. I think we crammed a year’s worth of social activity into a week’s worth of evenings and across the whole week, I almost got a whole night’s worth of sleep. There’s a test that goes something along the lines … Continue reading