I'm Adam. Nice to meet you.
My Twitter bio:
Metrics Lead at Mozilla Foundation. Previously at WWF. Test, tinker, and question the answers.
This post is an attempt to capture some of the things we’ve learned from a few busy and exciting weeks working on the Webmaker new user funnel. I will forget some things, there will be other stories to tell, and this will be biased towards my recurring message of “yay metrics”. How did this happen? As Dave pointed out in a recent email to Webmaker Dev list, “That’s a comma, not a decimal.” What happened to increase new user sign-ups by 1,024% compared the previous month? Is there one weird trick to…? No. Sorry, I know you’d like an easy answer… This growth is the result of a month of focused work and many many incremental improvements to the first-run experience for visitors arriving on webmaker.org from the promotion we’ve seen on the Firefox snippet. I’ll try to recount some … Continue reading
TL;DR: Check out this graph! Ever wondered how many Mozfest Volunteers also host events for Webmaker? Or how many code contributors have a Webmaker contributor badge? Now you can find out… The reason the MoFo Contributor dashboard we’re working from at the moment is called our interim dashboard is because it’s combining numbers from multiple data sources, but the number of contributors is not de-duped across systems. So if you’re counted as a contributor because you host an event for Webmaker, you will be double counted if you also file bugs in Bugzilla. And until now, we haven’t known what those overlaps look like. This interim solution wasn’t perfect, but it’s given us something to work with while we’re building out Baloo and the cross-org areweamillionyet.org (and by ‘we’, the vast credit for Baloo is due to our hard working … Continue reading
TL;DR Our MoFo dashboards now have trendlines based on known activity to date The recent uptick in activity is partly new contributors, and partly new recognition of existing contributors (all of which is good, but some of which is misleading for the trendline in the short term) Below is a rambling analogy for thinking about our contributor goals and how we answer the question ‘are we on track for 2014?’ + if you haven’t seen it, OpenMatt has crisply summarized a tonne of the data and insights that we’ve unpicked during Maker Party Stacking Logs I was stacking logs over the weekend, and wondering if I had enough for winter, when it struck me that this might be a useful analogy for a post I was planning to write. So bear with me, I hope this works… To be clear, … Continue reading
tl;dr: We’re wrapping up work on the MoFo Interim Dashboard The only other data source we’ll add is the badge counts for webmaker mentors & hive community members This is still our MoFo working document for the time being Then, we’ll switch our development efforts into integrating with Project Baloo Baloo is where we will de-dupe contributors across teams/tools etc. areweamillionyet.org will become our working document in time (‘time’ is TBC) Switching to Plan A The MoFo contributor dashboard we’ve been working with this year is our *interim* counting solution, and just as we’re “completing” it we’re now in a position to switch from an interim solution to a fully integrated system which is properly integrated with MoCo. This is pretty good timing, but it’s a change in scope for our immediate work so is worth a status update. Within … Continue reading
Here’s a happy update about our combined Mozilla Foundation (MoFo) and Mozilla Corporation (MoCo) contributor dashboards. TL;DR: There’s a demo All Mozilla Contributor Dashboard you can see at areweamillionyet.org It’s a demo, but it’s also real, and to explain why this is exciting might need a little context. Since January, I’ve been working on MoFo specific metrics. Mostly because that’s my job, but also because this/these organisations/projects/communities take a little while to understand, and getting to know MoFo was enough to keep me busy. We also wanted to ship something quickly so we know where we stand against our MoFo goals, even if the data isn’t perfect. That’s what we’ve built in our *interim* dashboard. It’s a non de-duped aggregation of the numbers we could get out of our current systems without building a full integration database. It gives us … Continue reading
While I’m always itching to get on with doing the work that needs doing, I’ve spent this morning writing about it instead. Part of me hates this, but another realizes this is valuable. Especially when you’re working remotely and the project status in your head is of no use to your colleagues scattered around the globe. So here’s the updated status page on our Mozilla Foundation Contributor Dashboard, and some progress on my ‘working open‘. Filing bugs, linking them to each other, and editing wiki pages can be tedious work (especially wiki tables that link to bugs!) but the end result is very helpful, for me as well as those following and contributing to the project. And a hat-tip to Pierros, whose hard-work on the project Baloo wiki page directly inspired the formatting here. Now, back to doing!
We’re seeing real progress getting data into the MoFo Contributor Dashboard now, but we need to keep in mind that counting existing contributors and engaging new contributors are two separate tasks that will help us move this line upwards. The gains we are seeing right now are counting gains rather than contribution gains. Getting this dashboard fully populated with our existing contributor numbers will be an achievement, but growing our contributor numbers is the real end goal of this work. 40-50% done? Using our back-of-the-napkin numbers from 2013 as a guide the current data sources shown on the dashboard today capture about 40% of the numbers we’re expecting to see here. Depending on how good our estimates were, and how many new contributors have joined in Q1, we expect this will be near the 5k mark by the time it’s … Continue reading
This is an idea for Webmaker teacher dashboards, and some thoughts on metadata related to learning analytics This post stems from a few conversations around metrics for Webmaker and learning analytics and it proposes some potential product features which need to be challenged and considered. I’m sharing the idea here as it’s easy to pass this around, but this is very much just an idea right now. For context, I’m approaching this from a metrics perspective, but I’m trying to solve the data gathering challenge by adding value for our users rather than asking them to do any extra work. These are the kind of questions I want us to be able to answer and that can inform future decision making in a positive way… How many people using Webmaker tools are mentors, students, or others? Do mentors teach many … Continue reading
Context: Last week I shared a few graphs (1, 2, 3, 4) looking at data from our repositories on Github, extracted using this Gitribution app thing, as part of our work to dashboard contributor numbers for the Mozilla Foundation. I didn’t comment on the graphs at the time because I wanted time for others to look at them without my opinions skewing what they might see. This follow up post is a walk-through of some things I see in the graphs/data. The real value in looking at data is finding ways to make things better by challenging ourselves, and being honest about what the numbers show, so this will be as much about questions as answers… Also, publishing this last week flagged up some missing repositories and identified some other members of staff so these graphs are based on the … Continue reading
Over the last week or so I’ve been building a thing: Gitribution. It’s an attempt to understand contributions to Mozilla Foundation work that happen on Github. It’s not perfect yet, but it’s in a state to get feedback on now. Why did I build this? For these reasons (in this order): Counting: To extract counts of contributor numbers from Github across Foundation projects on an automated ongoing basis Testing: To demo the API format we need for other sources of data to power our interim contributor dashboard Learning: To learn a bit about node.js so I can support metrics work on other projects more directly when it’s helpful to (i.e. submitting pull-requests rather than just opening bugs) 1. Counting The data in this tool is all public data from the Github API, but it’s been restructured so it can be … Continue reading