Webmaking in the UK, and face-to-face events

One of this week’s conversations was with Nesta, about Webmaker usage within the UK and whether or not we have data to support the theory that face-t0-face events have an impact getting people involved in making on the web. These are two topics that interest me greatly.

I’m basically copying some of my notes into blog form so that the conversation isn’t confined to a few in-boxes.

And the TL;DR is our data represents what we’ve done, rather than any universal truth.

Our current data would support the hypothesis that face-to-face time is important for learning, but that would simply be because that’s how our program has been designed to date. In other words, our Webmaker tools were designed primarily for use in face-to-face events, which meant that adoption by ‘self-learners’ online is low because their is little guidance or motivation to play with our tools on your own. This year we’re making a stronger push on developing tools that can be used remotely, alongside our work on volunteer led face-to-face events. This will lead to a less biased overall data set in the future where we can begin to properly explore the impact on making and learning for people who do or don’t attend face-to-face events at various stages in their learning experience. In particular I’m keen to understand what factors help people transition from learners, to mentoring and supporting their peers.

I also took a quick look at the aggregate Google Analytics location data for the UK audience which I hadn’t done before and which re-enforces the point above.

Screen Shot 2015-01-30 at 11.14.29

Above: Traffic to Webmaker (loosely indicating an interest in the topic) is roughly distributed like a population map of the UK. This is what I expect to see of most location data.

Screen Shot 2015-01-30 at 11.17.25

Above: However, if you look at the locations of visitors who make something, there are lots of clusters around the UK and London is equaled by many other cities.

To-date, usage of the Webmaker tools has been driven by those who are using the tools to teach the web (i.e. Webmaker Mentors). But we also know there are large numbers of people who find Webmaker outside of the face-to-face event scenarios who need a better route into Webmaker’s offering.

The good news is that this year’s plans look after both sets of potential learners.

#DALMOOC structure

I hesitantly post this, as I’m spending the evening looking at DALMOOC and hope to take part, but know I’m short on free time right now (what with a new baby and trying to buy a house) and starting the course late.

This is either the first in a series of blog posts about this course, or, we shall never talk about this again.

The course encourages open and distributed publishing of work and assessments, which makes answering this first ‘warm-up’ task feel like more of a commitment to the course than I can really make. But here goes…

Competency 0.1: Describe and navigate the distributed structure of DALMOOC, different ways to engage with peers and various technologies to manage and share personal learning.

DALMOOC offers and encourages learning experiences that span many online products from many providers but which all connect back to a core curriculum hosted on the edX platform. This ranges from learning to use 3rd party tools and software to interacting with peers on commercial social media platforms like Twitter and Facebook. Learners can pick the tools and engagement best suited to them, including an option to follow just the core curriculum within edX if they prefer to do so.

It actually feels a lot like how we work at Mozilla, which is overwhelming and disorientating at first but empowering in the long run.

Writing this publically, however lazily, has forced me to engage with the task much more actively than I might have just sitting back and watching a lecture and answering a quiz.

But I suspect that a fear of the web, and a lack of experience ‘working open’ would make this a terrifying experience for many people. The DALMOOC topic probably pre-selects for people with a higher than average disposition to work this way though, which helps.

Learning about Learning Analytics @ #Mozfest

If I find a moment, I’ll write about many of the fun and inspiring things I saw at Mozfest this weekend, but this post is about a single session I had the pleasure of hosting alongside Andrew, Doug and Simon; Learning Analytics for Good in the Age of Big Data.

We had an hour, no idea if anyone else would be interested, or what angle people would come to the session from. And given that, I think it worked out pretty well.

la_session

We had about 20 participants, and broke into four groups to talk about Learning Analytics from roughly 3 starting points (though all the discussions overlapped):

  1. Practical solutions to measuring learning as it happens online
  2. The ethical complications of tracking (even when you want to optimise for something positive – e.g. Learning)
  3. The research opportunities for publishing and connecting learning data

But, did anyone learn anything in our Learning Analytics session?

Well, I know for sure the answer is yes… as I personally learned things. But did anyone else?

I spoke to people later in the day who told me they learned things. Is that good enough?

As I watched the group during the session I saw conversations that bounced back and forth in a way that rarely happens without people learning something. But how does anyone else who wasn’t there know if our session had an impact?

How much did people learn?

This is essentially the challenge of Learning Analytics. And I did give this some thought before the session…

IMG_0184

As a meta-exercise, everyone who attended the session had a question to answer at the start and end. We also gave them a place to write their email address and to link their ‘learning data’ to them in an identifiable way. It was a little bit silly, but it was something to think about.

This isn’t good science, but it tells a story. And I hope it was a useful cue for the people joining the session.

Response rate:

  • We had about 20 participants
  • 10 returned the survey (i.e. opted in to ‘tracking’), by answering question 1
  • 5 of those answered question 2
  • 5 gave their email address (not exactly the same 5 who answered both questions)

Here is our Learning Analytics data from our session

Screen Shot 2014-10-30 at 13.53.26

Is that demonstrable impact?

Even though this wasn’t a serious exercise. I think we can confidently argue that some people did learn, in much the same way certain newspapers can make a headline out of two data points…

What, and how much they learned, and if it will be useful later in their life is another matter.

Even with the deliberate choice of question which was almost impossible to not show improvement from start to end of the session, one respondent claims to be less sure what the session was about after attending (but let’s not dwell on that!).

Post-it notes and scribbles

If you were at the session, and want to jog your memory about what we talked about. I kind-of documented the various things we captured on paper.

Screen Shot 2014-10-30 at 14.40.57
Click for gallery of bigger images

Into 2015

I’m looking forward to exploring Learning Analytics in the context of Webmaker much more in 2015.

And to think that this was just one hour in a weekend full of the kinds of conversations that repeat in your mind all the way until next Mozfest. It’s exhausting in the best possible way.