Learning about Learning Analytics @ #Mozfest

If I find a moment, I’ll write about many of the fun and inspiring things I saw at Mozfest this weekend, but this post is about a single session I had the pleasure of hosting alongside Andrew, Doug and Simon; Learning Analytics for Good in the Age of Big Data.

We had an hour, no idea if anyone else would be interested, or what angle people would come to the session from. And given that, I think it worked out pretty well.

la_session

We had about 20 participants, and broke into four groups to talk about Learning Analytics from roughly 3 starting points (though all the discussions overlapped):

  1. Practical solutions to measuring learning as it happens online
  2. The ethical complications of tracking (even when you want to optimise for something positive – e.g. Learning)
  3. The research opportunities for publishing and connecting learning data

But, did anyone learn anything in our Learning Analytics session?

Well, I know for sure the answer is yes… as I personally learned things. But did anyone else?

I spoke to people later in the day who told me they learned things. Is that good enough?

As I watched the group during the session I saw conversations that bounced back and forth in a way that rarely happens without people learning something. But how does anyone else who wasn’t there know if our session had an impact?

How much did people learn?

This is essentially the challenge of Learning Analytics. And I did give this some thought before the session…

IMG_0184

As a meta-exercise, everyone who attended the session had a question to answer at the start and end. We also gave them a place to write their email address and to link their ‘learning data’ to them in an identifiable way. It was a little bit silly, but it was something to think about.

This isn’t good science, but it tells a story. And I hope it was a useful cue for the people joining the session.

Response rate:

  • We had about 20 participants
  • 10 returned the survey (i.e. opted in to ‘tracking’), by answering question 1
  • 5 of those answered question 2
  • 5 gave their email address (not exactly the same 5 who answered both questions)

Here is our Learning Analytics data from our session

Screen Shot 2014-10-30 at 13.53.26

Is that demonstrable impact?

Even though this wasn’t a serious exercise. I think we can confidently argue that some people did learn, in much the same way certain newspapers can make a headline out of two data points…

What, and how much they learned, and if it will be useful later in their life is another matter.

Even with the deliberate choice of question which was almost impossible to not show improvement from start to end of the session, one respondent claims to be less sure what the session was about after attending (but let’s not dwell on that!).

Post-it notes and scribbles

If you were at the session, and want to jog your memory about what we talked about. I kind-of documented the various things we captured on paper.

Screen Shot 2014-10-30 at 14.40.57
Click for gallery of bigger images

Into 2015

I’m looking forward to exploring Learning Analytics in the context of Webmaker much more in 2015.

And to think that this was just one hour in a weekend full of the kinds of conversations that repeat in your mind all the way until next Mozfest. It’s exhausting in the best possible way.

My First #Mozfest

Mozfest 2013I have an hour free this morning, so wanted to quickly write up my thoughts on Mozfest before my memory fades too much. This will be a rough, but f*** it, ship it as they say at Mozfest.

I bought a Mozfest ticket in July with next to no expectations and just a little hope that meeting some new people might trigger some new ideas. It’s fair to say that this was a massive under-prediction on my part.

A couple of months later, with about a month to go until Mozfest, my boss (@ade) mentioned some sessions that might be interesting for WWF and my work in fundraising. A couple of introductory emails and a Skype call later and I’d put my name down for a yet-to-be-confirmed session called ‘Pass the App’.

We were going to use a new tool called Appmaker to build a donation app in a three hour session. At this point in time Appmaker didn’t do a lot. It was pre the version that would be called pre-alpha. I looked at Appmaker for a few minutes and worried I’d just agreed to waste the first quarter of Mozfest.

I had some time off and a couple of weeks went by. With two weeks to go, I wanted to get setup with Appmaker in a dev environment before the day so I didn’t waste people’s time with silly questions about configuration when we could be building things. Work was a bit crazier than usual and another week went by before I finally sat down to look at some code.

It was quite astounding how much Appmaker had evolved in those few weeks. The team working on this are incredible. From thinking the morning would be wasted, it now looked like a tool with enough components that with a little imagination you could hook up all sorts of awesome apps. My goal was to add some kind of payment to that.

The components in Appmaker are built with HTML, CSS and JavaScript and looking at a few examples I was happy I could build something by copying and adapting the work that’s already been done. But getting a development environment setup to work with these technologies I know pretty well required diving into a number of technologies that were completely new to me.

The deadline and motivation drove me through some of the initial hurdles and learning, and jumping into the IRC room for Appmaker I received great help and support from the team. I was worried about hassling them for their time while they were busy getting ready for Mozfest, but I was welcomed very warmly. It was a really great experience working with new people.

I guess the lesson here is: If you try and make something new, you cannot help but learn something new. And also that deadlines are amazing, as we’ve discussed before.

There were ten tracks at Mozfest, and at any given time I wanted to be in about eight of them. After the Saturday morning Pass the App session I was planning to alternate between the Open Data and Privacy tracks for lots of interesting things, but it didn’t work out that way. I didn’t actually make it to any other sessions. I got hooked into (and hooked on) making things in our scramble to build a working Pass the App demo, which we did. Here’s a link to the write up. I won’t re-tell that story. I got to work with kind and intelligent people making something valuable and learning a tonne. You can’t ask for more than that from any conference-esque event.

My hour of free time is up now, so I’m going to ship this despite the vast amount of things I was grateful for and wanted to talk about.

And I’ll say a quick hi to the people from the pass the app session,

And the many other lovely people I got to meet for the first time.