Deadlines and Decisions – an Introduction to Time Banking

Posted: May 9, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , | 3 Comments

I’m working on a new project, as part of my educational research, to change the way that students think about deadlines and time estimation. The concept’s called Time Banking and it’s pretty simple. Some schools already give students some ‘slack time’, free extension time that the students manage to allow them to manage their own deadlines. Stanford offers 2 days up front so, at any time in the course, you can claim some extra time and give yourself an extension.

The idea behind Time Banking is that you get extra hours if you hand up your work (to a certain standard) early. These hours can be used later as free extensions for assignment, up to some maximum number of days. This makes deadlines flexible and personalised per student.

Now I know that some of you already have your “Time is Money, Jones!” hats on and may even be waggling a finger. Here’s a picture of what that looks like, if you’re not a-waggling.

“Deadlines are fixed for a reason!”

“We use deadlines to teach professional conduct!”

“This is going to make marking impossible.”

“That’s not the right way to tie a bow tie!”

“It’s the end of civilisation as we know it!” (Sorry, that’s a little hyperbolic)

Of course, some deadlines are fixed. However, looking back over my own activities during the past quarter, I have far more negotiable and mutable deadlines than I do fixed ones. Knowing how to assess my own use of time in the face of a combination of fixed and mutable deadlines is a skill that I refine every year.

If I had up late, telling me to hand up on time or start earlier doesn’t really involve me in the process that’s required: making a decision as to how I’m going to manage all of my commitments over time, rather than panicking when I run into a deadline.

I can’t help thinking that forcing students to treat every assignment deadline as fixed, whether it needs to be or not, doesn’t deal with the student in the way that we try to in every other sphere. It makes them depend upon the deadline from an authority, rather than forcing them to look at their assignment work across a whole semester and plan inside that larger context. How can we produce students who are able to work at the multiplicity or commitment level, sorry, Perry again, if we force them to be authority-dependent dualists in their time management?

Now, before you think I’ve gone mad, there are some guidelines for all of this, as well as the requirement to have a good basis in evidence.

  1. We must be addressing an existing behavioural problem. (More on this later.)
  2. Some deadlines are immutable. This includes weekly dependencies, assignments where the solutions are revealed post submission, and ‘end of semester’ close-off dates.
  3. The assessment of ‘early and satisfactory’ must be low effort for the teacher. We don’t want to encourage handing up empty assignments a week ahead. We want to encourage meeting a certain standard, preferably automatically assessed, to bring student activity forward.
  4. We have limits on the amount you can bank or spend, to keep assessment of the submitted materials inside the realm of possibility and, again, to reduce unnecessary load on the staff,
  5. We don’t tolerate bad behaviour. Cheating or system fiddling immediately removes the system from the scheme.
  6. We provide up-front hours to give all students a base line of extension.
  7. We integrate this with our existing ‘system problem’ and ‘medical/compassionate problem’ extension systems.

Now, if students don’t have a problem, there’s nothing to fix. If our existing fixed deadline system encouraged students to start their work at the right time and finish in a timely fashion, then by final year, we wouldn’t need anything like this. However, my data from our web submission system clearly indicates the existence of ‘persistently’ late students and, in fact, rather than getting better, we actually start to see some students getting later in second, third and honours years. So, while this isn’t concrete, we’re not seeing the “Nope, no problem here” behaviour that we’d like. So that’s point 1 dealt with – it looks like we have a problem.

Most of the points are technical issues or components of an economic model, but 6 and 7 address a more important issue: equity. Right now, if your on-line submission systems crash the day before the assignment is due, what happens? Everyone who handed in their work has done the right thing but, because you have to grant a one day extension, they actually prioritised their work too early. Not a huge deal in many ways, because students who get their work in early probably march to a different drum anyway, but it makes a mockery of the whole fixed deadline thing. Either the deadline is fixed or it isn’t – by allowing extension on a broad scale for any reason, you’re admitting that your deadline was arbitrary.

We’re trying to make them think harder than that.

How about, instead, you hand out 24 hours of time in the bank. Now the students who handed up early have 24 hours to spend later on and the students who didn’t get it in before the crash have a fair chance to get their work in on time. Student gets sick, your medical extensions are now just managed as time in the bank, reflecting the fact that knock on effects can be far greater than just getting an extension for a single assignment.

But we don’t go crazy. My current thoughts are that we’d limit the students to only starting to count early about 2 days before the assignment is due, and allow a maximum of 3 days extension (greater for medical or compassionate). This keeps it in our marking boundary and also, assuming that you’ve placed your assignments in the context of the appropriate knowledge delivery, keeps the assignments roughly in the same location as the work – not doing the assignment at the beginning of the term and then forgetting the knowledge.

So, cards on the table, I’m writing a paper on this, identifying exactly what I need to look at in order to demonstrate if this is a problem, the literature that supports my approach, the objections to it and the obstacles. I also have to spec the technical system that would support it and , yes, identify the range of assignments for which it would work. It won’t work for everything/everyone or every course. But I suspect it might work very well for some areas.

Could we allow team banking? Course banking? Social sharing? Community involvement (donation to charity for so many hours in the bank at the end of the course)? What could we do by involving students in the elastic management of their own time?

There’s a lot more but I’d love to hear some thoughts on it. I look forward to the discussion!

Spot the Computer Science Student and Win!

Posted: May 8, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , | 1 Comment

CS Students get a pretty bad rap on that whole “stereotype” thing. Given that I’m an evidence-based researcher, let’s do some tests to find out if we can, in fact, spot the CS student. Here’s a quick game for you. Hidden in this image are 3 Computer Science students.

Which ones are they? (You can click on the image to enlarge it.)

I’ll make it easy for you to reference them – we’ll number the rows from the top (A) to the bottom (H) and the images from left to right as 1 to … well, whatever, because the rows aren’t the same length. So the picture with the cactus is A2, ok? Got it? Go!

Who did you pick? Got the details? Now scroll down.




























Of course, if you know me at all, you probably know the answer to this already.




































They’re ALL Computer Science students – well, they’re found in an image search for “I am a Computer Science student” and, while this is not guaranteed, it means that most of these students are in CS. Now, knowing that, go back and look at the ones you thought were music majors, physicists, business students, economics people. Yes, one or two of them probably look more likely than most but – wait for it – they don’t all look the same. Yeah, you know that, and I know that, but we just have to keep plugging away to make sure that everyone ELSE gets that. Heck, the pictures above are showing less pairs of glasses per person than you would expect from the average and there’s not even one light sabre! WON’T SOMEONE THINK OF THE STEREOTYPES???

This is only page 2 of the Image Search and I picked it because I liked the idea of some inanimate objects being labelled as CS students as well. Oh, that’s right, I said that you’d win something. You know never to trust me with statements unless I’m explicit in my use of terminology now. Sounds like a win to me!

(Of course, the guy with red hair is giving the strong impression that he now knows that you were looking at him on the Internet. I don’t know if you wanted that but that’s just how it is.)

Stats, stats and more stats: The Half Life of Fame

Posted: May 5, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , | 4 Comments

My WordPress stats for May 4th, 2012.

So, here are the stats for my blog, at time of writing. You can see a steady increase in hits over the last few weeks. What does this mean? Have I somehow hit a sweet spot in my L&T discussions? Has my secret advertising campaign paid off (no, not seriously). Well, there are a couple of things in there that are both informative… and humbling.

Firstly, two of the most popular searches that find my blog are “London 2012 tube” and “alone in a crowd”. These hits have probably accounted for about 16% of my traffic for the past three weeks. What does that tell me? Well, firstly, the Olympics aren’t too far away and people are looking for how convenient their hotels are. The second is a bit sadder.

The second search “alone in a crowd” is coming in across two languages – English and Russian. I have picked up a reasonable presence in Russia and Ukraine, mostly from that search term. It seems to contribute a lot to my (Australian) Monday morning feed, which means that a lot of people seem to search for this on Sundays.

But let me show you another graph, and talk about the half life of fame:

My WordPress stats, by week, on 4th of May, 2012.

That’s since the beginning my blogging activities. That spike at Week 9? That’s when I started blogging SIGCSE and also includes the day when over 100 people jumped on my blog because of a referral from Mark Guzdial. That was also the conference at which Hal Abelson referred to a concept of the Half Life of fame – the inevitable drop away after succeeding at something, if you don’t contribute more. And you can see that pretty clearly in the data. After SIGCSE, I was happily on my way back to being read by about 20-30 people a day, tops, most of whom I knew, because I wasn’t providing much more information to the people who scanned me at SIGCSE.

Without consciously doing it, I’ve managed to put out some articles that appear to have wider appeal and that are now showing up elsewhere. But these stats, showing improvement, are meaningless unless I really know what people are looking at. So, right now I’m pulling apart all of my log data to see what people are actually reading – whether I have an increasing L&T presence and readership, or a lot of sad Russian speakers or lost people on the London Underground system. I’m expecting to see another fall-away very soon now and drop down to the comfortable zone of my little corner of the Internet. I’m not interested in widespread distribution – I’m interesting in getting an inspiring or helpful message to the people who need it. Only one person needs to read this blog for it to be useful. It just has to the right one person. 🙂

One of the most interesting things about doing this, every day, is that you start wondering about whether your effort is worth it. Are people seeking it out? Are people taking the time to read it or just clicking through? Are there a growing number of frustrated Tube travellers thinking “To heck with Korzybski!” Time to go into the data and look. I’m going to keep writing regardless but I’d like to get an idea of where all of this is going.

Oh, Perry. (Our Representation of Intellectual Development, Holds On, Holds on.)

Posted: April 30, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , | 1 Comment

I’ve spent the weekend working on papers, strategy documents, promotion stuff and trying to deal with the knowledge that we’ve had some major success in one of our research contracts – which means we have to employ something like four staff in the next few months to do all of the work. Interesting times.

One of the things I love about working on papers is that I really get a chance to read other papers and books and digest what people are trying to say. It would be fantastic if I could do this all the time but I’m usually too busy to tear things apart unless I’m on sabbatical or reading into a new area for a research focus or paper. We do a lot of reading – it’s nice to have a focus for it that temporarily trumps other more mundane matters like converting PowerPoint slides.

It’s one thing to say “Students want you to give them answers”, it’s something else to say “Students want an authority figure to identify knowledge for them and tell them which parts are right or wrong because they’re dualists – they tend to think in these terms unless we extend them or provide a pathway for intellectual development (see Perry 70).” One of these statements identifies the problem, the other identifies the reason behind it and gives you a pathway. Let’s go into Perry’s classification because, for me, one of the big benefits of knowing about this is that it stops you thinking that people are stupid because they want a right/wrong answer – that’s just the way that they think and it is potentially possible to change this mechanism or help people to change it for themselves. I’m staying at the very high level here – Perry has 9 stages and I’m giving you the broad categories. If it interests you, please look it up!

A cloud diagram of Perry's categories.

(Image from geniustypes.com.)

We start with dualism – the idea that there are right/wrong answers, known to an authority. In basic duality, the idea is that all problems can be solved and hence the student’s task is to find the right authority and learn the right answer. In full dualism, there may be right solutions but teachers may be in contention over this – so a student has to learn the right solution and tune out the others.

If this sounds familiar, in political discourse and a lot of questionable scientific debate, that’s because it is. A large amount of scientific confusion is being caused by people who are functioning as dualists. That’s why ‘it depends’ or ‘with qualification’ doesn’t work on these people – there is no right answer and fixed authority. Most of the time, you can be dismissed as having an incorrect view, hence tuned out.

As people progress intellectually, under direction or through exposure (or both), they can move to multiplicity. We accept that there can be conflicting answers, and that there may be no true authority, hence our interpretation starts to become important. At this stage, we begin to accept that there may be problems for which no solutions exist – we move into a more active role as knowledge seekers rather than knowledge receivers.

Then, we move into relativism, where we have to support our solutions with reasons that may be contextually dependant. Now we accept that viewpoint and context may make which solution is better a mutable idea. By the end of this category, students should be able to understand the importance of making choices and also sticking by a choice that they’ve made, despite opposition.

This leads us into the final stage: commitment, where students become responsible for the implications of their decisions and, ultimately, realise that every decision that they make, every choice that they are involved in, has effects that will continue over time, changing and developing.

I don’t want to harp on this too much but this indicates one of the clearest divides between people: those who repeat the words of an authority, while accepting no responsibility or ownership, hence can change allegiance instantly; and those who have thought about everything and have committed to a stand, knowing the impact of it. If you don’t understand that you are functioning at very different levels, you may think that the other person is (a) talking down to you or (b) arguing with you under the same expectation of personal responsibility.

Interesting way to think about some of the intractable arguments we’re having at the moment, isn’t it?

Are you succeeding?

Posted: April 21, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , | Leave a comment

My wife sent me a link to this image, created by Alex Koplin and David Meiklejohn.

The message is (naively) simple – if you don’t like where you are, change something. This, of course, assumes that you have the capacity for change and the freedom to change. There are lots of times where this isn’t true but, in academia, we often have far more resources to hand to help people if they assess where they are going and don’t like the direction.

I talk a lot about process awareness – making students of what they are doing to ensure that they can identify the steps that they take and the impact that those steps have. My first-years have their first process awareness assignment to complete next week where I want them to look at their coding history in terms of difficulty and timeliness. What did they do that had a big impact on their chances of success? Being honest with themselves, were they lucky to get the work in on time? What I really want my students to understand is that they have to know enough about themselves and their capabilities that their work processes are:

  1. Predictable: They can estimate the time required to complete a task and the obstacles that they will encounter, and be reasonably accurate.
  2.  Reconfigurable: They can take apart their process to add new elements for new skills and re-use elements in new workflows.
  3. Well-defined and understood: Above all, they know what they are doing, why they are doing it and can explain it to other people.

Looking back at the diagram above, the most important step is change something if you don’t like where you are. By introducing early process awareness, before we ramp up programming difficulty and complexity, I’m trying to make my students understand the building blocks that they are using and, with this fundamental understanding, I hope that this helps them to be able to see what they could change, or even that change could be possible, if they need to try a different approach to achieve success.

Remember MIKE and SWEDE? Even a good student, who can usually pull off good work in a short time, may eventually be swamped by the scale of all the work that they have to do – without understanding which of their workflow components have to be altered, they’re guessing. Measurement of what works first requires understanding the individual elements. This are early days and I don’t expect anyone to be fully process aware yet, but I like the diagram, as it reminds me of why I’m teaching my students about all of this in the first place – to enable them to be active participants in the educational process and have the agency for change and the knowledge to change constructively and productively.

It’s Feedback Week! Yelling at Pilots is Good for Them!

Posted: April 4, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , | 1 Comment

The next few posts will deal with effective feedback and the impact of positive and negative reinforcement. I’ll start off with a story that I often tell in lectures.

There’s a well established fact among pilot instructors, in the Air Force, that negative reinforcement works better than positive reinforcement. Combat pilots  are very highly trained, go through an extremely rigorous selection procedure and have to maintain a very high level of personal discipline and fitness. The difference between success and failure, life and death effectively, can come down to one decision. During training, these pilots are put under constant stress, to try and prepare them for the real situation.

A pilot

This man redefines the phrase
"Highway to the Danger Zone".

The instructors have observed that these pilots respond better to being yelled at than being congratulated. What’s their evidence? Well, if a pilot does something really well, and is praised, chances are his or her performance will get worse. Whereas, if the pilot does something bad, or dangerous, and you yell at him or her, his or her performance will improve. Well, that’s a pretty simple framework. Let’s draw it as a diagram.

Now, of course, to see the impact of praise or abuse, you have to record what you did (the praise or abuse) and then what happened (improvement or decline). So we need to draw up some boxes. Ticks mean praise, crosses mean abuse. Up arrow mean improvement, down arrow means decline. A tick in the box that you reach by selecting the column that belongs to Praise (tick) and the row that belongs to Improvement (up arrow) means that the pilot improved after praise. Let’s look at a picture.

So here we can see that, much as an instructor would expect, when I’m nice to you, you get worse more often that you get better. You get better once when praised, compared to getting worse three times. But, wow, when I yelled at you, you get better far more often than you got worse!

There is, of course, a trick here. Yes, it appears that shouting works better than praising but, without giving the game away in the comments (or Googling), do you know why? (The data is a reasonable approximation of the real situation, so there are no hidden arrows or ticks anywhere. 🙂 )

Now, true confession, the picture above is not actually of pilot training – I don’t have an F18 on my desk – but it will be a reasonable approximation of the situation, although it comes from a different source. The graph above comes from a game called “Card Shouting” and I’ll tell you more about that in tomorrow’s post.

Improving, Holding Steady and Going Downhill – Giving Students Useful Feedback

Posted: April 3, 2012 | Author: | Filed under: Education | Tags: , , , , , , , | Leave a comment

I’ve written before about the slightly fuzzy nature of marks but that, overall, we can roughly class marks into ‘failing’, ‘doing okay’ and ‘doing really well’, One thing that I think is really useful is giving students an indication of how they are going in terms of getting better, staying where they are or falling behind.

This is hard throughout courses unless we’ve done some important things, and we’ve also committed to some things across an entire set of courses, like a degree. I’ve covered some of these before but this is a lot more focused.

  1. We’ve tied the assessment firmly into the course so that success in one aspect is a reasonable indicator of continued success.
  2. Students can get early indication when they’re not getting it – whether it’s quick quizzes, feedback on assignments or activities in lectures. Early warning signs are always there.
  3. WE follow up on the early warning signs as well to warn students of what’s going on.

It’s the whole instrumentation, measurement and action routine that I’ve been banging on about for three months now. But let’s put it into a tighter framework, in some senses, with a looser measurement system. We’re only worried about improvement, stability or decline. But is it ever that simple?

I honestly don’t really care if students get 87 or 90 in many ways – High Distinction is High Distinction. If a student gets a series of marks as 86, 91, 88, 90 they’re holding steady. I certainly don’t want them banging on my door, lamenting their decline, if their next mark is an 85 – this is stable and it’s good. But what about this sequence: 50, 52, 57, 53? This is a much riskier proposition, of course, because it’s so much closer to the fail line. Being under 55 is wandering into the zone where one bad mark or missed question could fail you. You don’t want to be stable here.

So, even with a simple three-way framework – it’s pretty obvious that stability is relative. What we really have is different zones where only some of these activities are valid, which should come as no surprise to anyone. 🙂

Below 60, stability doesn’t really cut it and decline is completely unacceptable. Below 60, you really want to be above 60. Yes, 50-60 is a pass but it’s also an indicator that you’re just scraping by – an unlucky day could cost you 6 months of work. Above 60, up to say 75? Stability is ok but we should really be aiming for improvement – if the student can. Above 75? Well, some people will never get much beyond that and all their striving will result in a hard-earned stability that may still taste a little bitter at times. This is where our knowledge of the student comes in.

Not knowing a student’s ability means that you risk telling them to improve when there’s nothing else left and they’ve given all they can. So let me throw out my classification framework and replace it with two questions for the student:

How do you feel about your mark?

Do you think you could have done better?

Balancing that with your knowledge of the student, and guiding them through the thinking process, will give them a better idea of what they can and can’t do. Did they really struggle to get that 66? No? Well, they could have worked harder and maybe got a better mark. That 75 nearly killed them and they really put their all into it? Well that’s one heck of a fine mark.

We all know that very few people know themselves and that’s why I like to try and help them understand why they’re doing what they’re doing and, maybe, once in a while, help them to either accept the fruits of their labours, or to strive that little bit more, if they still have something left to strive with and think it’s worthwhile.

Making Time, Taking Time: 70, 10, 10, 10

Posted: March 29, 2012 | Author: | Filed under: Education | Tags: , , , , , , | Leave a comment

I just finished reading Katrina’s post on students who are scared to interrupt us because we look so busy and it made a lot of sense. It’s certainly something I’ve struggled with and anyone who has come into my office in the past few months has seen that I am really trying to give everyone as much time as possible – but I’m obviously balancing a lot of things.

I’ve been toying with some new models for setting up my time for the day and something I’m finding that works is 70/10/10/10 time. I can lose up to 70% of my day with pre-scheduled appointments, lectures, tutes, meetings and things like that, but the remaining 30% is broken down like this:

  • 10% time reserved for the unforeseen – things like the opportunity to put a proposal in to attend an important meeting in California, that lobbed onto my desk yesterday and needed about 3 hours of work to get to fruition – completed by this Friday. I seem to get things like this every day!
  • 10% time reserved for me to do things like go to the bathroom, eat lunch and enjoy a coffee. I need time to get from point A to point B – and sitting the whole day hungry, thirsty or … anything else will not produce my best work.
  • 10% time, reserved in my head and on my calendar, for students who drop in to ask questions or who send me e-mail with questions (or post them on the forum). I should be making time. Yes, I have drop-in times normally but my students have a range of timetables and, after all, I am here to help. If I’m genuinely busy and out of time then I may have to use this time as well, but setting aside this time will help me to think about my students.

I look at the blog as an example. Every day, I put aside 20-30 minutes to write a post and, every day, I think of things that help me, or look for things to share, or go and do some research on CS Ed, or write up something interesting. (Some days I manage all of this!)  Putting time aside for something gives you the mental ability to think “Yes, this is important and I should do this.”

Thanks, Katrina!

 

The Map is not the Territory.

Posted: March 26, 2012 | Author: | Filed under: Education | Tags: , , , , , , | 4 Comments

I’m writing a paper on visualising Internet network topologies with my PhD student and some colleagues at the moment and an old friend, who is one of the student’s other supervisors, looked at some of the work we’d been doing and mentioned a great quote from Alfred Korzybski in 1931:

“The map is not the territory.”

Korzybski was a philosopher and scientist who developed the theory of general semantics, which I’m not going to talk more about here, but a lot of his work revolved around the idea that all we have access to is perceptions and beliefs, which we confuse with a knowledge of actual reality. This is a simple quote and a powerful concept: one of my favourite combinations.

What brought me to this was that, as part of our paper, we were looking at the London Underground map – the famous Tube Map.

A picture of the map of the London Underground

The focus of the Tube map is getting around London by Tube. Designed in 1931 by Harry Beck, a draughtsman with experience in laying out electrical circuits, it replaced a large number of incomplete and more geographically focused maps. What is most interesting about this map is that some licence is taken with the geography in order to make this the simplest map to use for Tube travel. Above ground, this map is not only not as useful, in some areas it’s completely wrong. (Suburbs on the opposite side shown, distances completely inaccurately represented for ease of reading.)

This has had an effect on the way that people travel around London – making decisions above ground that make sense on the Tube map but are downright silly when on foot on the streets. To combat this, Transport for London have developed the Legible London project with above-ground signage to assist the navigation of London Above, with signs and images showing you directions and landmarks.

Whether it’s maps of networks, maps of London or course pre-requisite diagrams, maps are only useful if you design them correctly for their primary use. Looking at the work on prerequisites that I’ve been talking about recently, it’s becoming more apparent that my desire for a good visualisation of pathways stems from my desire for a map that correctly reflects what we want students to do, reinforces the correct behaviour and is also going to be fit for purpose. Rather than using one diagram for many things, I need to check to make sure if I have the best diagram for a given situation.

Sometimes I need to release my grip on the accuracy of geography (precise location) to focus on the detail of topology (arrangement and connectivity). Sometimes it’s the other way around. Particularly when I insert a temporal aspect, I need to make sure that this “fourth dimension” doesn’t make my maps so complex that they’re useless. However, I always need a reason to relax a requirement: I’m certainly not saying that you can scribble randomly on a piece of paper and call it the NYC Subway map!

But, taking this concept further, how many pieces of work are out there that confuse a good diagram or a flowchart with the real thing? Is this just our confirmation of our perceptions and, as as result, it’s strongly sensible only when viewed from within our context? Or are we producing transferrable and shareable maps, focused on the right detail, showing the correct view of the terrain for the purpose, and accepting that there are an almost infinite set of views of the true territory?

A good map helps us to navigate territory but it can never replace it. What I always need to remember is that if I produce a map from a map, I can add no more detail than was in the original and I cannot correct mistakes in the original, without reference to the territory itself. And that’s something I think that is always worth remembering.

It’s okay to Karaoke (but why don’t you sing, instead?)

Posted: February 13, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , | 1 Comment

Today’s post focuses on the difference between memorising information and gaining knowledge. I don’t normally lead off with a descriptive sentence but I have to in this case because I’m going to talk about my previous life as a karaoke singer. I don’t want you to think I’ve accidentally posted on the wrong blog.

I quite enjoy singing and I have a reasonable karaoke voice – you’d never pay to hear me sing but you probably wouldn’t pay me to stop. When we used to sing in bars, when the machines first hit, you’d sing, people would listen, sometimes they’d clap and the assessment of the whole activity was based on how much you enjoyed it and, if you decided to compete, which of the dud t-shirts you won. For the record, my best placing was third place, a Yellow Cutty Sark t-shirt that made me look both jaundiced and leprous (a clever trick). I loved that t-shirt because, although third place was the best I ever achieved, but it was based on what the crowd felt and it was honestly earned. (I sang Prince’s Kiss in the style of Tom Jones, to complete this confession, so you may weigh the honesty for yourself.)

Of course, the fact that I can retrieve this fact, some 20 years down the track and well after the t-shirt was consigned to the rag pile, tells you a lot about the impression that this feedback had on me. However, the point of this anecdote is not that successful reinforcement is memorable (although that’s a good point) but that I wasn’t performing for points into a karaoke machine, I was using the technology to sing to subjective, analogue measuring devices – people, in other words. Now, the whole time I was singing, I had my own auditory feedback and (potentially) some crowd-based feedback but, being realistic, the only feedback I had was at the end if I won a t-shirt or if my friends either said ‘Yeah!’ or passed me a drink with a supportive expression. (Don’t judge me until you’ve attempted “Take On Me” by A-ha in a public space.)

Fast forward to today and you can have a karaoke machine in your home for the price of a PS2, PS3, X-Box or Wii. SingStar, Guitar Hero and Rock Band all provide you with the ability to play like a star and be adulated, and win awards, in the comfort of your own living room. Don’t like to rock out with pants on? Your lounge room – your rock and rules, baby! How, in this context, do we provide guidance that what you’re doing matches the song on screen? How do I award you a yellow Cutty Sark t-shirt?

In most cases, there is a ‘correct’ interpretation of the musical line – based on tone for singing and choice of input pad and timing for the other instruments in rock band. If you perform the right action (with varying degrees of tolerance) at the right time (again, with tolerance) you are recorded as having done the right thing. Based on this you get points, awards, more opportunities. There is nothing intrinsically wrong with this – this is a valid feedback mechanism. The problem arises when you don’t regard the score as assessing the activity, but the activity as a means for maximising the score.

I was slightly surprised the first time I saw Rock Band being played by a group and the singer, instead of singing, mumbled the tones into the microphone, at a volume I could barely hear, in order to exactly match what was going across his screen without the risk of missing the note or stumbling by getting a word wrong. Sure enough, excellent score. I’ve seen people ace SingStar while singing out of tune because they completely repeated every phrasing, every tonal movement and every pause, but stayed within the tolerance, despite it being nearly painful to the human ear. Why were they still singing? Look at the score! I’m doing it right! I bet I can make a higher score if I sing the same song another 10,000 times!

At the same time, I’ve seen professional singers get canned by SingStar and given terrible scores, despite putting together a beautiful performance of the song. What’s going on?

Anyone who’s played any of these games probably knows why this is: SingStar (and RockBand to a great extent) reward memorisation and recall, not interpretation. It’s the degree to which you can recite the song that matters, not your knowledge of how the song is constructed and how it can be rearranged, while still being that song – or better.

Musically speaking, it takes a reasonable amount of experience and knowledge to be able to start interpreting songs in a way that sounds good. Anyone can hit vibrato and wave around the song like a 90s boy band on a roller-coaster – it takes talent to harmonise, built, manipulate cadence and involve the audience in something that makes them breathless. It takes even more talent to know that you don’t do that all the time but in the right place and at the right time. SingStar has no easy way to assess anything outside of the norm so it will reward you if you hit note X at time Y within tolerance Delta. If you decide to mess with the cadence or add some colour, which are both demonstrations of a sound knowledge of the underlying work and the techniques that are valid for manipulating it, you won’t do well. This makes sense from an electronic game designer’s perspective – you can’t encode all of musical theory into the game but you can easily check for conformity.

(To its credit, Rock Band does allow some room for free-styling on certain instruments and at certain times. However, I think it’s probably good that Keith Moon is dead or he would have personally picked up every kit and eaten it after his first encounter with that particular game.)

It should come as no surprise that I believe that we often fall into the trap of requiring memorisation and recitation, effectively in lieu of demonstration of knowledge. Recited answers are easy to mark, manually and automatically. Knowledge requires the marker to be knowledgeable – subtleties abound and there are fewer templates. Electronic systems often lean in this direction for exactly the same reason as SingStar. A simple script to check the output of something is far easier to write than a detailed analyser with all human knowledge in this area.

Knowledge has to be built upon a strong foundation of information – there have to be core facts to provide a basis for your learning (such as the multiplication tables or the names of human anatomy, as two obvious examples). But at the higher level, at the 2nd and 3rd year level of University, we shouldn’t be rewarding people purely for mumbling our own words back to us in the microphone. We’re after knowledge and the demonstration of knowledge and that means more than training parrots.

A bit of karaoke never hurt anyone, but we in higher ed should be training our students to interpret to the fullest, to learn everything and to be able to show us what they know in ways that surprise, delight and, yes, challenge us.

They should be singing.

← Older Entries
Newer Entries →