The Many Types of Failure: What Does Zero Mean When Nothing Is Handed Up?

You may have read about the Edmonton, Canada, teacher who expected to be sacked for handing out zeros. It’s been linked to sites as diverse as Metafilter, where a long and interesting debate ensued, and Cracked, where it was labelled one of the ongoing ‘pussifications’ of schools. (Seriously? I know you’re a humour site but was there some other way you could have put that? Very disappointed.)

Basically, the Edmonton Public School Board decided that, rather than just give a zero for a missed assignment, this would be used as a cue for follow-up work and additional classes at school or home. Their argument – you can’t mark work that hasn’t been submitted, let’s use this as a trigger to try and get submission, in case the source is external or behavioural. This, of course, puts the onus on the school to track the students, get the additional work completed, and then mark out of sequence. Lynden Dorval, the high school teacher who is at the centre of this, believe that there is too much manpower involved in doing this and that giving the student a zero forces them to come to you instead.

Some of you may never have seen one of these before. This is a zero, which is the lowest mark you can be awarded for any activity. (I hope!)

Now, of course, this has split people into two fairly neat camps – those who believe that Dorval is the “hero of zero” and those who can see the benefit of the approach, including taking into account that students still can fail if they don’t do enough work. (Where do I stand? I’d like to know a lot more than one news story before I ‘pick a side’.) I would note that a lot of tired argument and pejorative terminology has also come to the fore – you can read most of the buzzwords used against ‘progressives’ in this article, if you really want to. (I can probably summarise it for you but I wouldn’t do it objectively. This is just one example of those who are feting Dorval.)

Of course, rather than get into a heated debate where I really don’t have enough information to contribute, I’d rather talk about the basic concept – what exactly does a zero mean? If you hand something in and it meets none of my requirements, then a zero is the correct and obvious mark. But what happens if you don’t hand anything in?

With the marking approach that I practice and advertise, which uses time-based mark penalties for late submission, students are awarded marks for what they get right, rather than have marks deducted for what they do wrong. Under this scheme, “no submission” gives me nothing to mark, which means that I cannot give you any marks legitimately – so is this a straight-forward zero situation? The time penalties are in place as part of the professional skill requirements and are clearly advertised, and consistently policed. I note that I am still happy to give students the same level of feedback on late work, including their final mark without penalty, which meets all of the pedagogical requirements, but the time management issues can cost a student some, most or all of their marks. (Obviously, I’m actively working on improving engagement with time management through mechanisms that are not penalty based but that’s for other posts.)

As an aside, we have three distinct fail grades for courses at my University:

  • Withdraw Fail (WF), where a student has dropped the course but after the census date. They pay the money, it stays on their record, but as a WF.
  • Fail (F), student did something but not enough to pass.
  • Fail No Submission (FNS), student submitted no work for assessment throughout the course.

Interestingly, for my Uni, FNS has a numerical grade of 0, although this is not shown on the transcript. Zero, in the course sense, means that you did absolutely nothing. In many senses, this represents the nadir of student engagement, given that many courses have somewhere from 1-5, maybe even 10%, of marks available for very simple activities that require very little effort.

My biggest problem with late work, or no submission, is that one of the strongest messages I have from that enormous data corpus of student submission that I keep talking about is that starting a pattern of late or no submission is an excellent indicator of reduced overall performance and, with recent analysis, a sharply decreased likelihood of making it to third year (final year) in your college studies. So I really want students to hand something in – which brings me to the crux of the way that we deal with poor submission patterns.

Whichever approach I take should be the one that is most likely to bring students back into a regular submission pattern. 

If the Public School Board’s approach is increasing completion rates and this has a knock-on effect which increases completion rates in the future? Maybe it’s time to look at that resourcing profile and put the required money into this project. If it’s a transient peak that falls off because we’re just passing people who should be failing? Fuhgeddaboutit.

To quote Sherlock Holmes (Conan Doyle, naturally): 

It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts. (A Scandal in Bohemia)

“Data! Data! Data!” he cried impatiently. “I can’t make bricks without clay.” (The Adventure of the Copper Beeches)

It is very easy to take a side on this and it is very easy to see how both sides could have merit. The issue, however, is what each of these approaches actually does to encourage students to submit their assignment work in a more timely fashion. Experiments, experimental design, surveys, longitudinal analysis, data, data, data!

If I may end by waxing lyrical for a moment (and you will see why I stick to technical writing):

If zeroes make Heroes, then zeroes they must have! If nulls make for dulls, then we must seek other ways!


Rush, Rush: Baby, Please Plan To Submit Your Work Earlier Than The Last Minute

Sorry, Paula Abdul, but I had to steal a song lyric from you.

AND MANGLE IT!

This is the one of the first pictures that comes up when you search for ‘angry Paula Abdul”. Sorry, Lamar.

I’ve been marking the first “process awareness” written report from my first-year students. A one-page PDF that shows their reflections on their timeliness and assignment performance to date and how they think that they can improve it or maintain it. There have been lots of interesting results from this. From about 100 students, I’ve seen many reports along the lines of “I planned, I assigned time, SO WHY DIDN’T I FOLLOW THE PLAN?” or “Wow, I never realised how much I needed a design until I was stuck in the middle of a four-deep connection of dynamic arrays.”

This is great – understanding why you are succeeding or failing allows you to keep doing the things that work, and change the things that don’t. Before this first-year curriculum restructure, and this course, software development process awareness could avoid our students until late second- or third-year. Not any more. You got run over by the infamous Library prac? You know, you should have written a design first. And now my students have all come to this realisation as well. Two of my favourite quotes so far are:

“[Programming in C++] isn’t hard but it’s tricky.”

and

“It’s not until you have a full design [that you can] see the real scope of the project.”

But you know I’m all about measurement so, after I’d marked everything, I went back and looked at the scores, and the running averages. Now here’s the thing. The assignment was marked out of 10. Up until 2 hours before the due date, the overall average was about 8.3. For the last two hours, the average dropped to 7.2. The people commenting in the last two hours were making loose statements about handing up late, and not prioritising properly, but giving me enough that I could give them some marks. (It’s not worth a lot of marks but I do give marks for style and reflection, to encourage the activity.) The average mark is about 8/10 usually. So, having analysed this, I gave the students some general feedback, in addition to the personalised feedback I put on every assignment, and then told them about that divide.

The fact that the people before the last minute had the marks above the average, and that the people at the last minute had the marks below.

One of the great things about a reflection assignment like this is that I know that people are thinking about the specific problem because I’ve asked them to think about it and rewarded them with marks to do so. So when I give them feedback in this context and say “Look – planned hand-in gets better marks on average than last-minute panic” there is a chance that this will get incorporated into the analysis and development of a better process, especially if I give firm guidelines on how to do this in general and personalised feedback. Contextualisation, scaffolding… all that good stuff.

There are, as always, no guarantees, but moving this awareness and learning point forward is something I’ve been working on for some time. In the next 10 days, the students have to write a follow-up report, detailing how they used the lessons they learnt, and the strategies that they discussed, to achieve better or more consistent results for the next three practicals. Having given them guidance and framing, I now get to see what they managed to apply. There’s a bit of a marking burden with this one, especially as the follow-up report is 4-5 pages long, but it’s worth it in terms of the exposure I get to the raw student thinking process.

Apart from anything else, let me point out that by assigning 2/10 for style, I appear to get reports at a level of quality where I rarely have to take marks away and they are almost all clear and easy to read, as well as spell-checked and grammatically correct. This is all good preparation and, I hope, a good foundation for their studies ahead.


The Confusing Message: Sourcing Student Feedback

Once, for a course which we shall label ‘an introduction to X and Y’, I saw some feedback from a student that went as follows. A single student, on the same feedback form, and in adjacent text boxes, gave these answers:

What do you like most about this course: the X

What would you like to see happen to improve the course: less X, more Y!

Now, of course, this not inherently contradictory but, honestly, it’s really hard to get the message here. You think that X is great but less useful than Y, although you like X more? You’re a secret masochist and you like to remove pleasure from your life?

As (almost) always, the problem here is that we these two questions, asked in adjacent text boxes, are asking completely different things. Survey construction is an art, a dark and mysterious art, and a well-constructed survey will probably not answer a question once, in one way. It will ask the same question in multiple ways, sometimes in the negative, to see if the “X” and “not ( not (X))” scores line up for each area of interest. This, of course, assumes that you have people who are willing to fill out long surveys and give you reliable answers. This is a big assumption. Most of the surveys that I work with have to fit into short time frames and are Likert-based with text boxes. Not quite yes/no tick/flick but not much more and very little opportunity for mutually interacting questions.

The Red Tick of Courage!

Our student experience surveys are about 10 questions long with two text boxes and are about the length that we can fit into the end of a lecture and have the majority of students fill out and return. From experience, if I construct larger surveys, or have special ‘survey-only’ sessions, I get poor participation. (Hey, I might just be doing it wrong. Tips and help in the comments, please!)

Of course, being Mr Measurement, I often measure things as side effects of the main activity. Today, I held a quiz in class and while everyone was writing away, I was actually getting a count of attendees because they were about to hand up cards for marking. This gives me an indicator of attendance and, as it happens, two weeks away from the end of the course, we’re still getting good attendance. (So, I’m happy.) I can also see how the students are doing with fundamental concepts so I can monitor that too.

I’m fascinated by what students think about their experience but I need to know what they need based on their performance, so that I can improve their performance without having to work out what they mean. The original example would give me no real insight into what to do and how to improve – so I can’t really do anything with any certainty. If the student had said “I love X but I feel that we spent too much time on it and it could be just as good with a little less.” then I know what I can do.

I also sometimes just ask for direct feedback in assignments, or in class, because then I’ll get the things that are really bugging or exciting people. That also gives me the ability to adapt to what I hear and ask more directed questions.

Student opinion and feedback can be a vital indicator of our teaching efficacy, assuming that we can find out what people think rather than just getting some short and glib answers to questions that don’t really probe in the right ways, where we never get a real indication of their thoughts. To do this requires us to form a relationship, to monitor, to show the value of feedback and to listen. Sadly, that takes a lot more work than throwing out a standard form once a semester, so it’s not surprising that it’s occasionally overlooked.


If We’re Going To Measure, Let’s Measure Properly: Teaching Isn’t a NASCAR Race

I’ve been reading a Huff Post piece on teacher assessment, entitled “Carolyn Abbott, The Worst 8th Grade Math Teacher In New York City, Victim Of Her Own Success”, where a teacher, Carolyn Abbot, at a gifted and talented school in Manhattan was rated being the worst teacher in 8th grade.

The problem, it appears, is the measurement used where your contribution is based upon whether your students have performed better or worse than last year on the Teacher Data Report, a measure used to assess contribution to English and Math. So here’s the problem. The teacher taught maths to grades 7 and 8 and her Grade 7 students achieved at the 98th percentile for their test in 2009. Therefore, according to the Teacher Data Reports modelling process, the same students should have achieved 97th percentile in their Grade 8 tests the following year. They only managed 89th percentile. Abbot had made a significant negative contribution to her students, by this logic, and her ranking was the lowest in NYC 8th grade mathematics teachers.

Yes, you read that right. She’s kept the students in the 1.5-2 standard deviations above the norm category. The students have moved up a year and are now starting to run into the puberty zone, always fun, they’re still scoring in the 89th percentile – and she’s the worst teacher in NYC. Her students struggle with the standardised testing itself: the non-mathematical nature of the tests, the requirement to put in a single answer when the real answer is potentially more complex, the fact that multiple choice can be trained for (rather than test anything) – and they’re still kicking out at the +85 level. Yet, she’s the worst 8th grade math teacher in NYC.

This also goes against one of my general principles of assessment, in that the performance of someone else affects the assessment of your performance. (Yes, that leaves me at odds with national testing schemes, because I don’t see a way that they can be meaningfully calibrated across many different teaching systems and economic influences. It’s obvious that New York haven’t worked it out properly for one system and one economy!) Having a notion of acceptable and unacceptable is useful here. Having a notion of exemplary, acceptable and unacceptable is useful here. Having a notion of best and worst is meaningless, because all these teachers could score 100/100 and one scores 100/99 and they’re the worst. Ranking must be combined with standards of acceptability where professional practice is required. This isn’t a NASCAR race: in teaching, everyone can cross the line in a way that they win.

I am a big fan of useful, carefully constructed and correctly used measurement but this story is an example of what happens if you come up with a simple measure that gives you a single number that isn’t much use but is used as if it means something. Now, if over time, you saw a large slide in scores from one teacher and that dropped down low enough, then maybe this number would mean something but any time that you simple number has to come with an explanation – it’s not that simple anymore.

In this case, what’s worse is that the rankings were published with names. Names of teachers and names of schools. Abbott’s boss reassured her that he would still put her up for tenure but felt he had to warn her that someone else might take these rankings into account.

Abbott’s ranking doesn’t matter to her much anymore, because this teacher has now left teaching and is undertaking a PhD in Mathematics instead. Great for us at University because having good teachers who then successfully complete PhDs often works out very well – they’re highly desirable employees in many ways. Not so good for the students at her school who have been deprived of a teacher who managed to get a group of kids to the 98th percentile on Grade 7 Math, providing a foundation that will probably be with them for their whole lives (even if we quibble and it’s down to the 89th) and giving them a better start for their academic future.

But that’s ok, kids, because she was the worst teacher you’d ever have. Oh, of course, there’s another new “worst” teacher because that’s how our ranking system works. Sorry about that. Good luck, Carolyn Abbott!


Stats, stats and more stats: The Half Life of Fame

My WordPress stats for May 4th, 2012.

So, here are the stats for my blog, at time of writing. You can see a steady increase in hits over the last few weeks. What does this mean? Have I somehow hit a sweet spot in my L&T discussions? Has my secret advertising campaign paid off (no, not seriously). Well, there are a couple of things in there that are both informative… and humbling.

Firstly, two of the most popular searches that find my blog are “London 2012 tube” and “alone in a crowd”. These hits have probably accounted for about 16% of my traffic for the past three weeks. What does that tell me? Well, firstly, the Olympics aren’t too far away and people are looking for how convenient their hotels are. The second is a bit sadder.

The second search “alone in a crowd” is coming in across two languages – English and Russian. I have picked up a reasonable presence in Russia and Ukraine, mostly from that search term. It seems to contribute a lot to my (Australian) Monday morning feed, which means that a lot of people seem to search for this on Sundays.

But let me show you another graph, and talk about the half life of fame:

My WordPress stats, by week, on 4th of May, 2012.

That’s since the beginning my blogging activities. That spike at Week 9? That’s when I started blogging SIGCSE and also includes the day when over 100 people jumped on my blog because of a referral from Mark Guzdial. That was also the conference at which Hal Abelson referred to a concept of the Half Life of fame – the inevitable drop away after succeeding at something, if you don’t contribute more. And you can see that pretty clearly in the data. After SIGCSE, I was happily on my way back to being read by about 20-30 people a day, tops, most of whom I knew, because I wasn’t providing much more information to the people who scanned me at SIGCSE.

Without consciously doing it, I’ve managed to put out some articles that appear to have wider appeal and that are now showing up elsewhere. But these stats, showing improvement, are meaningless unless I really know what people are looking at. So, right now I’m pulling apart all of my log data to see what people are actually reading – whether I have an increasing L&T presence and readership, or a lot of sad Russian speakers or lost people on the London Underground system. I’m expecting to see another fall-away very soon now and drop down to the comfortable zone of my little corner of the Internet. I’m not interested in widespread distribution – I’m interesting in getting an inspiring or helpful message to the people who need it. Only one person needs to read this blog for it to be useful. It just has to the right one person. 🙂

One of the most interesting things about doing this, every day, is that you start wondering about whether your effort is worth it. Are people seeking it out? Are people taking the time to read it or just clicking through? Are there a growing number of frustrated Tube travellers thinking “To heck with Korzybski!” Time to go into the data and look. I’m going to keep writing regardless but I’d like to get an idea of where all of this is going.


Are you succeeding?

My wife sent me a link to this image, created by Alex Koplin and David Meiklejohn.

The message is (naively) simple – if you don’t like where you are, change something. This, of course, assumes that you have the capacity for change and the freedom to change. There are lots of times where this isn’t true but, in academia, we often have far more resources to hand to help people if they assess where they are going and don’t like the direction.

I talk a lot about process awareness – making students of what they are doing to ensure that they can identify the steps that they take and the impact that those steps have. My first-years have their first process awareness assignment to complete next week where I want them to look at their coding history in terms of difficulty and timeliness. What did they do that had a big impact on their chances of success? Being honest with themselves, were they lucky to get the work in on time? What I really want my students to understand is that they have to know enough about themselves and their capabilities that their work processes are:

  1. Predictable: They can estimate the time required to complete a task and the obstacles that they will encounter, and be reasonably accurate.
  2.  Reconfigurable: They can take apart their process to add new elements for new skills and re-use elements in new workflows.
  3. Well-defined and understood: Above all, they know what they are doing, why they are doing it and can explain it to other people.

Looking back at the diagram above, the most important step is change something if you don’t like where you are. By introducing early process awareness, before we ramp up programming difficulty and complexity, I’m trying to make my students understand the building blocks that they are using and, with this fundamental understanding, I hope that this helps them to be able to see what they could change, or even that change could be possible, if they need to try a different approach to achieve success.

Remember MIKE and SWEDE? Even a good student, who can usually pull off good work in a short time, may eventually be swamped by the scale of all the work that they have to do – without understanding which of their workflow components have to be altered, they’re guessing. Measurement of what works first requires understanding the individual elements. This are early days and I don’t expect anyone to be fully process aware yet, but I like the diagram, as it reminds me of why I’m teaching my students about all of this in the first place – to enable them to be active participants in the educational process and have the agency for change and the knowledge to change constructively and productively.


Improving, Holding Steady and Going Downhill – Giving Students Useful Feedback

I’ve written before about the slightly fuzzy nature of marks but that, overall, we can roughly class marks into ‘failing’, ‘doing okay’ and ‘doing really well’, One thing that I think is really useful is giving students an indication of how they are going in terms of getting better, staying where they are or falling behind.

This is hard throughout courses unless we’ve done some important things, and we’ve also committed to some things across an entire set of courses, like a degree. I’ve covered some of these before but this is a lot more focused.

  1. We’ve tied the assessment firmly into the course so that success in one aspect is a reasonable indicator of continued success.
  2. Students can get early indication when they’re not getting it – whether it’s quick quizzes, feedback on assignments or activities in lectures. Early warning signs are always there.
  3. WE follow up on the early warning signs as well to warn students of what’s going on.

It’s the whole instrumentation, measurement and action routine that I’ve been banging on about for three months now. But let’s put it into a tighter framework, in some senses, with a looser measurement system. We’re only worried about improvement, stability or decline. But is it ever that simple?

I honestly don’t really care if students get 87 or 90 in many ways – High Distinction is High Distinction. If a student gets a series of marks as 86, 91, 88, 90 they’re holding steady. I certainly don’t want them banging on my door, lamenting their decline, if their next mark is an 85 – this is stable and it’s good. But what about this sequence: 50, 52, 57, 53? This is a much riskier proposition, of course, because it’s so much closer to the fail line. Being under 55 is wandering into the zone where one bad mark or missed question could fail you. You don’t want to be stable here.

So, even with a simple three-way framework – it’s pretty obvious that stability is relative. What we really have is different zones where only some of these activities are valid, which should come as no surprise to anyone. 🙂

Below 60, stability doesn’t really cut it and decline is completely unacceptable. Below 60, you really want to be above 60. Yes, 50-60 is a pass but it’s also an indicator that you’re just scraping by – an unlucky day could cost you 6 months of work. Above 60, up to say 75? Stability is ok but we should really be aiming for improvement – if the student can. Above 75? Well, some people will never get much beyond that and all their striving will result in a hard-earned stability that may still taste a little bitter at times. This is where our knowledge of the student comes in.

Not knowing a student’s ability means that you risk telling them to improve when there’s nothing else left and they’ve given all they can. So let me throw out my classification framework and replace it with two questions for the student:

How do you feel about your mark?

Do you think you could have done better?

Balancing that with your knowledge of the student, and guiding them through the thinking process, will give them a better idea of what they can and can’t do. Did they really struggle to get that 66? No? Well, they could have worked harder and maybe got a better mark. That 75 nearly killed them and they really put their all into it? Well that’s one heck of a fine mark.

We all know that very few people know themselves and that’s why I like to try and help them understand why they’re doing what they’re doing and, maybe, once in a while, help them to either accept the fruits of their labours, or to strive that little bit more, if they still have something left to strive with and think it’s worthwhile.


Summer Camps for Computer Students: Sustainable, Effective and Replicable

 

Great post over on Mark Guzdial’s blog on the work being done by Barbara Ericson on Computing Summer Camps. You should head on over and read it (not you, Mark, but thank you!) but the core message is so useful and transferable that I wanted to reiterate it here. Student activities that foster engagement, participation and skill development are very popular but, to be successful, you have to make sure that you do them right. I had a chance to see Barb present when she and Mark were in Adelaide and her talk was really helpful because it was informative but also really, really useful. Too many times I’ve seen people talk a great theory at me but without giving me any starting points. Her talk, and the paper, highlight good practices with a strong basis. Here are the three points that capture why Barb’s summer camps are so good – with my own commentary added somewhat superfluously.

  • Effectiveness is essential. Measuring student performance is vital to showing that students do improve – in attitudes and knowledge. If the camp isn’t effective in either increasing engagement or driving knowledge, then why are we bothering? I’m, going to mention MIKE again here – Measurement Is the Key to Everything. If we don’t measure, we have no idea what has succeeded or how we can make it work.
  • The program is sustainable and will keep going after the first flush of money runs out. This is an enormous problem with so many of the programs I’ve seen – they work beautifully while the big cash is available and disappear when it dries up. Barb’s Summer Camps are sustainable as a whole because she’s done this for long enough to get some great rules of thumb for keeping enough money in from key groups to allow an investment in slightly smaller groups (such as using a large residential middle school camp to offset the costs of a smaller high school camp).
  •  The camps can be run by other people and still be successful. This replicability is another thing that’s frequently missing from our courses. All of my materials should be able to survive me moving on but, too often, they come close but I don’t quite capture all of the details – although I strive to. Barb’s aim is to have these programs running lots of places and, by making the material available and providing seed grants, there are now 11 more camps around Georgia, returning similar results in terms of success.  There is only one Barb, but at the moment we have a 12-fold increase in ‘BarbCamposity’ through scaling. If Computer Scientists should be good at anything, it’s leveraging amplifiers to allow us to be in more than one place at once.

There are so many other places we can apply these principles and, most importantly, it identifies the focus of our efforts as educators – I don’t want my students to need me all the time, I want to bring information to them that is sound, that extends them and that supports them for years to come. By making sure that my material is effective, identifying needs, measuring impacts, I avoid wasting my time. By developing sustainable programs, which aren’t resource heavy, I can keep going whether we’re getting big dollars, small dollars, no dollars or (shudder) negative dollars as we slash budgets to ride out troubled times. Finally, my making my course so self-contained and good that someone else can teach it and someone else WANTS to teach it, I can go on to the next thing I want to do. This is liberating – I’m not writing myself out of a job, I’m giving myself the scope to pursue new techniques, to share my knowledge (such as it is) with other educators and to spend my time where it’s most needed.

Please go and read the blog post, and the paper, because both are really good and I’m only shadowing them here. One of the things I love about the vitality of the CSE community is that I can interact with, and learn so much from, people like Mark and Barb, but also share it with you – efficiently, sustainably and (given that I’m reflagging) in a replicable manner.


I Am Thinking, HE/SHE Is Procrastinating, THEY Are Daydreaming

This is a follow-up thought to my recent post on laziness. I spend a lot of time thinking and, sometimes, it would be easy to look at me and think “Wow, he’s not doing anything.” Sometimes, in my office, I stare at a wall, doodle, pace the corridor, sketch on the whiteboard or, if I’m really stuck, go for a walk down by the river. All of this helps me to clear and organise my thoughts. I use tools to manage what I have to do and to get it done in time but the cognitive work of thinking things through sometimes takes time. The less I sleep, the longer it takes. That’s why, while I’m jet lagged, I will do mostly catch-up and organisational work rather than thinking. Right now I can barely do a crossword, which is an excellent indicator that my brain is fried for anything much more complex than blogging. Given that I last slept in a bed over 30 hours ago, this isn’t surprising.

Now it’s easy to accept that I stumble around, somewhat absent-mindedly, because I’m an academic and you can all understand that my job requires me to do a lot of thinking…

But so many jobs require a lot of thinking to be done well – or , at least, the component tasks that go to make up modern jobs.

It’s a shame then that it’s activity that most people focus on rather than quality. If I were to sit in my office and type furiously but randomly, answer mails curtly, and never leave for coffee or cake, have to schedule meetings three weeks in advance – what a powerhouse I would appear! Except, of course, that I wouldn’t really appear to be that to people who knew what I was supposed to do. I don’t do the kind of job where I can move from task to task without, in most cases, detailed research including a search for new material, construction, creation, design, analysis, building, testing and executing. As always, this doesn’t make my job better or worse than anyone else’s, but I don’t carry out the same action repeatedly, an action that can be reduced in cognitive load with familiarity, I tend to do something at least slightly different each time. Boiler plate repetition is more likely to indicate that I am not doing my job correctly, given the roles that I hold.

So, if there are no points in a week where I sit there with books or papers or doodles or sketches of ideas and I think about them – then I’m really running the risk of not doing my job. I need to produce work of high quality and, because there’s a lot of new content creation, there’s creation/editing/testing… load throughout. Some of which, to an external viewer, looks like sitting around throwing paper into the bin while I hunt for solutions.

I think about this a lot for my students. I expect them, in a lecture, to not sit and think so much that they don’t communicate. I will try and bring them back from mental flights of fancy rather than let them fly off because I’ve only got an hour or two with them and need to try to get certain concepts across. And then what? Sometime in 4th year, or PhD, I expect them to flip a switch and realise that the apparent inactivity of quiet, contemplative thought is one of the most productive activities? That a day where you write eight pages, and on review only salvage half of one page, could be the most important and useful day in your PhD?

This is why I tend not to give out marks for ‘just anything’ – two pages of nonsense gets zero, there are no marks for effort because I am rewarding the wrong activity, especially where we haven’t achieved quality. Similarly, I don’t give out marks for attendance but for the collaboration – if you are after an activity, getting the students to do something, I think it’s always best to reward them for doing the activity, not just attending the framing session! But this, of course, comes hand-in hand with the requirement to give them enough timely feedback that they can improve their mark – by improving the quality of what they produce.

Electronic learning systems could be really handy here. Self-paced learning, with controlled remote assessment mechanisms, allows this thinking time and the ability to sit, privately, and mull over the problems. Without anyone harassing them.

Years ago, when I was still in the Army Reserve, we were on exercise for a couple of weeks and my soldiers were getting pretty tired because we’d been running 4 hour shifts to staff the radios. You sat on the radios for 4 hours, you were off for 4. Every so often you might get 6 hours off but it was unlikely. This meant that my soldiers were often sleeping in the middle of the day, desperately trying to make up lost sleep as well as periodically showering, shaving and eating. 4 hours goes really quickly when you’re not on duty. People in our base area who WEREN’T doing these shifts thought that my soldiers were lazy and, on at least two occasions, tried to wake them up to use them on work parties – digging holes, carrying things, doing soldier stuff. My soldiers needed their sleep and I was their commander so I told the other people, politely, to leave them alone. My operators had a job to do and maintained the quality of their work by following a very prescribed activity pattern – but the people around them could only see inactivity because of their perspective.

Maybe it’s time to look at my students again, look at what I’m asking them to do and make sure that what I’m asking and that the environment I’m giving them is the right one. I don’t think we’re doing too badly, because of previous reviews, but it’s probably never too soon to check things out again.


SIGCSE, Why Can’t I Implement Everything?

I was going to blog about Mike Richards’ excellent paper on ubicomp, but Katrina did a much better job so I recommend that you go and look over there.

My observation on this session are more feeling based, in that I’ve seen many things at this conference and almost every time, I’ve wanted to tell more people about it, or adopt the mechanism. As Katrina said to me, when we were discussing it over lunch, you can’t do everything because we only have so many people and not every idea has to be implemented at every University.

But it’s such a shame! I want small home-rolled mobile computing platforms and fascinating programming environments! Everything good I saw, I want to bring home and share with people. However, the hard part is that I want them to be as fascinated and as excited as I am – and they’re getting it from me second-hand.

The other things that I have to remember is that whatever we do, we have to commit to and do well, we can’t just bring stuff in, try it and throw it away in case there’s a possibility of our ad-hoc approach hurting our students. We have to work out what we want to improve, measure it, try the change and then measure it again to see what has changed.

You’ll see a few more SIGCSE posts, because there’s still some very interesting things to report and comment on, but an apparent movement away from the content here isn’t a sign that I’ve stopped thinking about – it’s a sign that I’m thinking about which bits I can implement and which bits I have to put into the ‘long term’ box to bring up at a strategy level down the track.

I’ve met a lot of great people and heard many wonderful things – thanks to everyone at SIGCSE!