I'm Adam. Nice to meet you.
My Twitter bio:
Metrics Lead at Mozilla Foundation. Previously at WWF. Test, tinker, and question the answers.
TL;DR: Check out this graph! Ever wondered how many Mozfest Volunteers also host events for Webmaker? Or how many code contributors have a Webmaker contributor badge? Now you can find out… The reason the MoFo Contributor dashboard we’re working from at the moment is called our interim dashboard is because it’s combining numbers from multiple data sources, but the number of contributors is not de-duped across systems. So if you’re counted as a contributor because you host an event for Webmaker, you will be double counted if you also file bugs in Bugzilla. And until now, we haven’t known what those overlaps look like. This interim solution wasn’t perfect, but it’s given us something to work with while we’re building out Baloo and the cross-org areweamillionyet.org (and by ‘we’, the vast credit for Baloo is due to our hard working … Continue reading
TL;DR Our MoFo dashboards now have trendlines based on known activity to date The recent uptick in activity is partly new contributors, and partly new recognition of existing contributors (all of which is good, but some of which is misleading for the trendline in the short term) Below is a rambling analogy for thinking about our contributor goals and how we answer the question ‘are we on track for 2014?’ + if you haven’t seen it, OpenMatt has crisply summarized a tonne of the data and insights that we’ve unpicked during Maker Party Stacking Logs I was stacking logs over the weekend, and wondering if I had enough for winter, when it struck me that this might be a useful analogy for a post I was planning to write. So bear with me, I hope this works… To be clear, … Continue reading
We’re a little over halfway through the year now, and our dashboard is now good enough to tell us how we’re doing. TL;DR: The existing trend lines won’t get us to our 2014 goals but knowing this is helpful and getting there is possible Ask less: How do we count our contributors? Ask more: What are we doing to grow the contributor community? And, are we on track? Changing the question Our dashboard now needs to move from being a project to being a tool that helps us do better. After all, Mozilla’s unique strength is that we’re a community of contributors and this dashboard, and the 2014 contributor goal, exist to help us focus our workflows, decisions and investments in ways that empower the community. Not just for the fun of counting things. The first half of the year … Continue reading
tl;dr: We’re wrapping up work on the MoFo Interim Dashboard The only other data source we’ll add is the badge counts for webmaker mentors & hive community members This is still our MoFo working document for the time being Then, we’ll switch our development efforts into integrating with Project Baloo Baloo is where we will de-dupe contributors across teams/tools etc. areweamillionyet.org will become our working document in time (‘time’ is TBC) Switching to Plan A The MoFo contributor dashboard we’ve been working with this year is our *interim* counting solution, and just as we’re “completing” it we’re now in a position to switch from an interim solution to a fully integrated system which is properly integrated with MoCo. This is pretty good timing, but it’s a change in scope for our immediate work so is worth a status update. Within … Continue reading
Here’s a happy update about our combined Mozilla Foundation (MoFo) and Mozilla Corporation (MoCo) contributor dashboards. TL;DR: There’s a demo All Mozilla Contributor Dashboard you can see at areweamillionyet.org It’s a demo, but it’s also real, and to explain why this is exciting might need a little context. Since January, I’ve been working on MoFo specific metrics. Mostly because that’s my job, but also because this/these organisations/projects/communities take a little while to understand, and getting to know MoFo was enough to keep me busy. We also wanted to ship something quickly so we know where we stand against our MoFo goals, even if the data isn’t perfect. That’s what we’ve built in our *interim* dashboard. It’s a non de-duped aggregation of the numbers we could get out of our current systems without building a full integration database. It gives us … Continue reading
I presented a version of this on the Mozilla Foundation staff call yesterday, and thought it’s worth a write-up for those who weren’t on the call and those in MoCo working on related things. Some Context: One of the cross Mozilla goals this year is to “10X” the number of active contributors, with a longer term goal of growing to a million Mozillians. When the 10X goal was set we weren’t really sure what X was, for valid reasons; defining contribution is as much art as it is science, and the work plans for this year include building the tools to measure this in a systematic way. The goals justify the tools, and vice versa. Chicken and egg. 2,000 contributors were invited to the summit, so the target was set at 20k active contributors shared between MoCo and MoFo. MoFo … Continue reading
We’re seeing real progress getting data into the MoFo Contributor Dashboard now, but we need to keep in mind that counting existing contributors and engaging new contributors are two separate tasks that will help us move this line upwards. The gains we are seeing right now are counting gains rather than contribution gains. Getting this dashboard fully populated with our existing contributor numbers will be an achievement, but growing our contributor numbers is the real end goal of this work. 40-50% done? Using our back-of-the-napkin numbers from 2013 as a guide the current data sources shown on the dashboard today capture about 40% of the numbers we’re expecting to see here. Depending on how good our estimates were, and how many new contributors have joined in Q1, we expect this will be near the 5k mark by the time it’s … Continue reading
I’ve just updated the main wiki page tracking our contributor dashboard project, so I won’t repeat everything here. The quick update is that the puzzle pieces that will make our interim contributor dashboard work are coming together now. Which means we have a live dashboard front-end, (with a few data-holes we need to plug!). This screenshot is just data from Github. Let’s gather some more numbers…
This is an idea for Webmaker teacher dashboards, and some thoughts on metadata related to learning analytics This post stems from a few conversations around metrics for Webmaker and learning analytics and it proposes some potential product features which need to be challenged and considered. I’m sharing the idea here as it’s easy to pass this around, but this is very much just an idea right now. For context, I’m approaching this from a metrics perspective, but I’m trying to solve the data gathering challenge by adding value for our users rather than asking them to do any extra work. These are the kind of questions I want us to be able to answer and that can inform future decision making in a positive way… How many people using Webmaker tools are mentors, students, or others? Do mentors teach many … Continue reading
TL;DR: Share your thoughts on the language we use around contribution metrics here (anyone can contribute): https://etherpad.mozilla.org/contributors-dashboard-language Then if you have the time, here are some of my thoughts on this topic… What does language have to do with metrics? You’d be forgiven for thinking that working with data and metrics is a clean and scientific-like process of running queries against a database or two and generating a report. In many ways, I’m glad it’s not as simple as that. Metrics are only as good as the things they enable us to improve. Which means while metrics need to be grounded in good clean data, they are primarily for people; and not just for people to read. In their best incarnation, metrics motivate people to change things for the better. At this scale, motivating people is definitely more art than … Continue reading