This is an idea for Webmaker teacher dashboards, and some thoughts on metadata related to learning analytics
This post stems from a few conversations around metrics for Webmaker and learning analytics and it proposes some potential product features which need to be challenged and considered. I’m sharing the idea here as it’s easy to pass this around, but this is very much just an idea right now.
For context, I’m approaching this from a metrics perspective, but I’m trying to solve the data gathering challenge by adding value for our users rather than asking them to do any extra work.
These are the kind of questions I want us to be able to answer
and that can inform future decision making in a positive way…
- How many people using Webmaker tools are mentors, students, or others?
- Do mentors teach many times?
- How many learners go on to become mentors?
- What size groups do mentors typically work with?
- How many mentors teach once, and then never again? (their feedback would be very useful)
- How many learners come back to Webmaker tools several days after a lesson?
- Which partnership programme reached the greatest number of learners?
And the particularly tricky area…
- What data points show developing competencies in Web Literacy?
Flexible and organic data points to suit the Webmaker ecosystem
The Webmaker suite of tools are very open and flexible and as a result get used by people for many different things. Which personally, I like a lot. However, this also makes understanding our users more difficult.
When looking at the data, how can we tell if a new Thimble Make has come from a teacher, a student, or even an experienced web developer who works at Mozilla and is using the tool to publish their birthday wishes to the web? The waters here are muddy.
We need a few additional indicators in the data to analyze it in a meaningful way, but these indicators have to work with the informal teaching models and practices that exist in the Webmaker ecosystem.
On the grounds that everyone has both something to teach and to learn, and that we want trainers to train trainers and so on, I propose that asking people to self-identify as mentors via a survey/check-box/preferences/etc will not yield accurate flags in the data.
The journey to identifying yourself as a mentor is personal and complex, and though that process is immensely interesting, there are simpler things we can measure.
The simplest measure is that someone who teaches something is a teacher. That sounds obvious, but it’s very slightly different from someone who thinks of themselves as a teacher.
If we build a really useful tool for teaching (I’m suggesting one idea below) and its use identifies Webmaker accounts as teacher(s) and/or learner(s) then we’d have useful metadata to answer almost all of those questions asked above.
When we know who the learners are we can better understand what learning looks like in terms of data (a crucial step in conversations about learning analytics).
If anyone can use this proposed tool as part of their teaching process, and students can engage with it as students. Then anyone can teach, or attend a lesson in any order without having to update their account records to say “I first attended a Maker Party, then I taught a session on remixing for the web, and now I’m learning about CSS and next I want to teach about Privacy”.
A solution like this doesn’t need 100% use by all teachers and learners to be useful (which helps the solution remain flexible if it doesn’t suit). It just needs enough people to use it to use it that we have a meaningful sample of Webmaker teachers and learners flagged in the database.
With a decent sample we can see what teaching with Webmaker looks like at scale. And with this kind of data, continually improve the offering.
An idea: ‘Teacher Lesson Dashboards’
I think Teacher Lesson Dashboards would catch the metadata we need, and I’ll sketch this out here. Don’t get stuck on any naming I’ve made up right now, the general process for the teacher and the learner is the main thing to consider.
1. Starting with a teacher/mentor
User logs in to Webmaker.org
Clicks an option to “Create a new Lesson”
Gets an interface to ‘build-up’ a Lesson (a curation exercise)
Adds starter makes to the lesson (by searching for their own and/or others makes)
e.g. A ‘Lesson’ might include:
- A teaching kit with discussion points, and a link to X-ray goggles demo
- A thimble make for students to remix
- A (deliberately) broken thimble make for students to try and debug
- A popcorn make to remix and report back what they have learned
They give their lesson a name
Add optional text and an image for the lesson
Save their new Lesson, and get a friendly short URL
Then point students to this at the beginning of the teaching session
2. The learner(s) then…
Go the URL the mentor provides
Optionally, check-in to the lesson (and create a Webmaker account at the same time if required)
Have all the makes and activities they need in one place to get started
One click to view or remix any make in the Lesson
Can reference any written text to support the lesson
3. Then, going back to the mentor
Each ‘Lesson’ also has a dashboard showing:
- Who has checked-in to the lesson
- with quick links to their most recent makes
- links to their public profile pages
- Perhaps integrating together.js functionality if you’re running a lesson remotely?
- Metrics that help with teaching (this is a whole other conversation, but it depends first on being able to identify who is teaching who)
- Feedback on future makes created after the lesson (i.e. look what your session led to further down the line)
4. And to note…
‘Lessons’ as a kind of curated make, can also me remixed and shared in some way.
I’m not on the front-lines using the tools right now, so this is a proposal very much from a person who wants flags in a database 🙂
- Does this feel like it adds value to mentors and/or learners?
- Do you think is a good way to identify who’s teaching and who’s learning? (and who’s doing both of course)
— Adam Lofting (@adamlofting) March 11, 2014