2015 Mozilla Foundation Metrics Strategy(ish) & Roadmap(ish)

I wrote a version of this strategy in January but hadn’t published it as I was trying to remove those ‘ish‘s from the title. But the ‘ish’ is actually a big part of my day-to-day work, so this version embraces the ‘ish’.

MoFo Metrics Measures of Success:

These are ironically, more qualitative than quantitative.

  1. Every contributor (paid or volunteer) knows at any given time what number they (or we) are trying to move, where that number is right now, and how they hope to influence it.
  2. We consider metrics (i.e. measures of success) before, during and after after each project.
  3. We articulate the stories behind the metrics we aim for, so their relevance isn’t lost in the numbers.
  4. A/B style testing practice has a significant impact on the performance of our ‘mass audience’ products and campaigns.

1. Every contributor (paid or volunteer) knows at any given time what number they (or we) are trying to move, where that number is right now, and how they hope to influence it.

  • “Every” is ambitious, but it sets the right tone.
  • This includes:
    • Public dashboards, like those at https://metrics.webmaker.org
    • Updates and storytelling throughout the year
    • Building feedback loops between the process, the work and the results (the impact)

2. We consider metrics (i.e. measures of success) before, during and after after each piece of work.

  • This requires close integration into our organizational planning process
  • This work is underway, but it will take time (and many repetitions) before it becomes habit

3. We articulate the stories behind the metrics we aim for, so their relevance isn’t lost in the numbers.

  • The numbers should be for navigation, rather than fuel

4. A/B style testing practice has a significant impact on the performance of our ‘mass audience’ products and campaigns.

  • This is the growth hacking part of the plan
  • We’ve had some successes (e.g. Webmaker and Fundraising)
  • This needs to become a continuous process

Those are my goals.

In many cases, the ultimate measure of success is when this work is done by the team rather than by me for the team.

We’re working on Process AND Culture

Process and culture feed off of and influence each other. Processes must suit the culture being cultivated. A data driven culture can blinker creativity – it doesn’t have to, but it can. And a culture that doesn’t care for data, won’t care for processes related to data. This strategy aims to balance the needs of both.

A roadmap?

I tried to write one, but basically this strategy will respond to the roadmaps of each of the MoFo teams.

So, what does Metrics work look like in 2015?

  • Building the tools and dashboards to provide the organisational visibility we need for our KPIs
  • ‘Instrumenting’ our products so that we can accurately measure how they are being used
  • Running Optimization experiments against high profile campaigns
  • Running training and support for Google Analytics, Optimizely, and other tools
  • Running project level reporting and analysis to support iterative development
  • Consulting to the Community Development Team to plan experimental initiatives

Plus: supporting teams to implement our data practices, and of course, the unknown unknowns.

…ish

One thought on “2015 Mozilla Foundation Metrics Strategy(ish) & Roadmap(ish)”

Comments are closed.