At least they’re being honest

I was inspired to write this by a comment about using late penalties but dealing slightly differently with students when they owned up to being late. I have used late penalties extensively (it’s school policy) and so I have a lot of experience with the many ways students try to get around them.

6874balance_scale

Like everyone, I have had students who have tried to use honesty where every other possible way of getting the assignment in on time (starting early, working on it before the day before, miraculous good luck) has failed. Sometimes students are puzzled that “Oh, I was doing another assignment from another lecturer” isn’t a good enough excuse. (Genuine reasons for interrupted work, medical or compassionate, are different and I’m talking about the ambit extension or ‘dog ate my homework’ level of bargaining.)

My reasoning is simple. In education, owning up to something that you did knowing that it would have punitive consequences of some sort should not immediately cause things to become magically better. Plea bargaining (and this is an interesting article of why that’s not a good idea anywhere) is you agreeing to your guilt in order to reduce your sentence. But this is, once again, horse-trading knowledge on the market. Suddenly, we don’t just have a temporal currency, we have a conformal currency, where getting a better deal involves finding the ‘kindest judge’ among the group who will give you the ‘lightest sentence’. Students optimise their behaviour to what works or, if they’re lucky, they have a behaviour set that’s enough to get them to a degree without changing much. The second group aren’t mostly who we’re talking about and I don’t want to encourage the first group to become bargain-hunting mark-hagglers.

I believe that ‘finding Mr Nice Lecturer’ behaviour is why some students feel free to tell me that they thought someone else’s course was more important than mine, because I’m a pretty nice person and have a good rapport with my students, and many of my colleagues can be seen (fairly or not) as less approachable or less open.

We are not doing ourselves or our students any favours. At the very least, we risk accusations of unfairness if we extend benefits to one group who are bold enough to speak to us (and we know that impostor syndrome and lack of confidence are rife in under-represented groups). At worst, we turn our students into cynical mark shoppers, looking for the easiest touch and planning their work strategy based on what they think they can get away with instead of focusing back on the learning. The message is important and the message must be clearly communicated so that students try to do the work for when it’s required. (And I note that this may or may not coincide with any deadlines.)

We wouldn’t give credit to someone who wrote ‘True’ and then said ‘Oh, but I really meant False’. The work is important or it is not. The deadline is important or it is not. Consequences, in a learning sense, do not have to mean punishments and we do not need to construct a Star Chamber in our offices.

Yes, I do feel strongly about this. I completely understand why people do this and I have also done this before. But after thinking about it at length, I changed my practice so that being honest about something that shouldn’t have happened was appreciated but it didn’t change what occurred unless there was a specific procedural difference in handling. I am not a judge. I am not a jury. I want to change the system so that not only do I not have to be but I’m not tempted to be.


Ugliness 101: late punishments

Before I lay out the program design I’m thinking of (and, beyond any discussion of competency, as a number of you have suggested, we are heading towards Bloom’s mastery learning as a frame with active learning elements), we need to address one of the most problematic areas of assessment.

Late penalties.

Well, let’s be accurate, penalties are, by definition, punishments imposed for breaking the rules, so these are punishments. This is the stick in the carrot-and-stick reward/punish approach to forcing people to do what you want.

Let’s throw the Greek trinity at this and see how it shapes up. A student produces an otherwise perfect piece of work for an assessment task. It’s her own work. She has spent time developing it. It’s really good. Insightful. Oh, but she handed it up a day late. So we’re now going to say that this knowledge is worth less because it wasn’t delivered on time. She’s working a day job to pay the bills? She should have organised herself better. No Internet at home? Why didn’t she work in the library? I’m sure the campus is totally safe after hours and, well, she should just be careful in getting to and from the library. After all, the most important thing in her life, without knowing anything about her, should be this one hundred line program to reinvent something that has been written over a million times by every other CS student in history.

A picture of an owl with orange eyes, staring at the viewer.

Have an owl, while you think about that.

That’s not truth. That’s establishing a market value for knowledge with a temporal currency. To me, unless there’s a good reason for doing this, this is as bad as curve grading because it changes what the student has achieved for reasons outside of the assignment activity itself.

“Ah!” you say “Nick, we want to teach people to hand work in on time because that’s how the world works! Time is money, Jones!

Rubbish. Yes, there are a (small) number of unmovable deadlines in the world. We certainly have some in education because we have to get grades in to achieve graduations and degrees. But most adults function in a world where they choose how to handle all of the commitments in their lives and then they schedule them accordingly. The more you do that, the more practice you get and you can learn how to do it well.

If you have ever given students a week, or even a day’s, extension because of something that has stopped you being able to accept or mark student work, no matter how good the reason, you have accepted that your submission points are arbitrary. (I feel strongly about this and have posted about it before.)

So what would be a good reason for sticking to these arbitrary deadlines? We’d want to see something really positive coming out of the research into this, right? Let’s look at some research on this, starting with Britton and Tesser, “Effects of Time-Management Practices on College Grades”, J Edu Psych, 1991, 83, 3. This reinforces what we already know from Bandura: students who feel in control and have high self-efficacy are going to do well. If a student sits down every day to work out what they’re going to do then they, unsurprisingly, can get things done. But this study doesn’t tell us about long-range time planning – the realm of instrumentality, the capability to link activity today with success in the future. (Here are some of my earlier thoughts on this, with references to Husman.) From Husman, we know that students value tasks in terms of how important they think it is, how motivated they are and how well they can link future success to the current task.

In another J Edu Psych paper (1990,82,4), Macan and Shahani reported that participants who felt that they had control over what they were doing did better but also clearly indicated that ambiguity and stress had an influence on time management in terms of perception and actuality. But the Perceived Control of Time (author’s caps) dominated everything, reducing the impact of ambiguity, reducing the impact of stress, and lead to greater satisfaction.

Students are rarely in control of their submission deadlines. Worse, we often do not take into account everything else in a student’s life (even other University courses) when we set our own deadlines. Our deadlines look arbitrary to students because they are, in the majority of cases. There’s your truth. We choose deadlines that work for our ability to mark and to get grades in or, perhaps, based on whether we are in the country or off presenting research on the best way to get students to hand work in on-time.

(Yes, the owl above is staring at me just as hard as he is staring at anyone else here.)

My own research clearly shows that fixed deadlines do not magically teach students the ability to manage their time and, when you examine it, why should it? (ICER 2012, was part of a larger study that clearly demonstrated students continuing, and even extending, last-minute behaviour all the way to the fourth year of their studies.) Time management is a discipline that involves awareness of the tasks to be performed, a decomposition of those tasks to subtasks that can be performed when the hyperbolic time discounting triggers go off, and a well-developed sense of instrumentality. Telling someone to hand in their work by this date OR ELSE does not increase awareness, train decomposition, or develop any form of planning skills. Well, no wonder it doesn’t work any better than shouting at people teaches them Maxwell’s Equations or caning children suddenly reveals the magic of the pluperfect form in Latin grammar.

So, let’s summarise: students do well when they feel in control and it helps with all of the other factors that could get in the way. So, in order to do almost exactly the opposite of help with this essential support step, we impose frequently arbitrary time deadlines and then act surprised when students fall prey to lack of self-confidence, stress or lose sight of what they’re trying to do. They panic, asking lots of (what appear to be) unnecessary questions because they are desperately trying to reduce confusion and stress. Sound familiar?

I have written about this at length while exploring time banking, giving students agency and the ability to plan their own time, to address all of these points. But the new lens in my educational inspection loupe allows me to be very clear about what is most terribly wrong with late penalties.

They are not just wrong, they satisfy none of anyone’s educational aesthetics. Because we don’t take a student’s real life into account, we are not being fair. Because we are not actually developing the time management abilities but treating them as something that will be auto-didactically generated, we are not being supportive. Because we downgrade work when it is still good, we are being intellectually dishonest. Because we vary deadlines to suit ourselves but may not do so for an individual student, we are being hypocritical. We are degrading the value of knowledge for procedural correctness. This is hideously “unbeautiful”.

That is not education. That’s bureaucracy. Just because most of us live within a bureaucracy doesn’t mean that we have to compromise our pedagogical principles. Even trying to make things fit well, as Rapaport did to try and fit into another scale, we end up warping and twisting our intent, even before we start thinking about lateness and difficult areas such as that. This cannot be good.

There is nothing to stop a teacher setting an exercise that is about time management and is constructed so that all steps will lead someone to develop better time management. Feedback or marks that reflect something being late when that is the only measure of fitness is totally reasonable. But to pretend that you can slap some penalties on to the side of an assessment and it will magically self-scaffold is to deceive yourself, to your students’ detriment. It’s not true.

Do I have thoughts on how to balance marking resources with student feedback requirements, elastic time management, and real assessments while still recognising that there are some fixed deadlines?

Funny you should ask. We’ll come back to this, soon.


EduTech AU 2015, Day 2, Higher Ed Leaders, “Assessment: The Silent Killer of Learning”, #edutechau @eric_mazur

No surprise that I’m very excited about this talk as well. Eric is a world renowned educator and physicist, having developed Peer Instruction in 1990 for his classes at Harvard as a way to deal with students not developing a working physicist’s approach to the content of his course. I should note that Eric also gave this talk yesterday and the inimitable Steve Wheeler blogged that one, so you should read Steve as well. But after me. (Sorry, Steve.)

I’m not an enormous fan of most of the assessment we use as most grades are meaningless, assessment becomes part of a carrot-and-stick approach and it’s all based on artificial timelines that stifle creativity. (But apart from that, it’s fine. Ho ho.) My pithy statement on this is that if you build an adversarial educational system, you’ll get adversaries, but if you bother to build a learning environment, you’ll get learning. One of the natural outcomes of an adversarial system is activities like cheating and gaming the system, because people start to treat beating the system as the goal itself, which is highly undesirable. You can read a lot more about my views on plagiarism here, if you like. (Warning: that post links to several others and is a bit of a wormhole.)

Now, let’s hear what Eric has to say on this! (My comments from this point on will attempt to contain themselves in parentheses. You can find the slides for his talk – all 62MB of them – from this link on his website. ) It’s important to remember that one of the reasons that Eric’s work is so interesting is that he is looking for evidence-based approaches to education.

Eric discussed the use of flashcards. A week after Flashcard study, students retain 35%. After two weeks, it’s almost gone. He tried to communicate this to someone who was launching a cloud-based flashcard app. Her response was “we only guarantee they’ll pass the test”.

*low, despairing chuckle from the audience*

Of course most students study to pass the test, not to learn, and they are not the same thing. For years, Eric has been bashing the lecture (yes, he noted the irony) but now he wants to focus on changing assessment and getting it away from rote learning and regurgitation. The assessment practices we use now are not 21st century focused, they are used for ranking and classifying but, even then, doing it badly.

So why are we assessing? What are the problems that are rampant in our assessment procedure? What are the improvements we can make?

How many different purposes of assessment can you think of? Eric gave us 90s to come up with a list. Katrina and I came up with about 10, most of which were serious, but it was an interesting question to reflect upon. (Eric snuck

  1. Rate and rank students
  2. Rate professor and course
  3. Motivate students to keep up with work
  4. Provide feedback on learning to students
  5. Provide feedback to instructor
  6. Provide instructional accountability
  7. Improve the teaching and learning.

Ah, but look at the verbs – they are multi-purpose and in conflict. How can one thing do so much?

So what are the problems? Many tests are fundamentally inauthentic – regurgitation in useless and inappropriate ways. Many problem-solving approaches are inauthentic as well (a big problem for computing, we keep writing “Hello, World”). What does a real problem look like? It’s an interruption in our pathway to our desired outcome – it’s not the outcome that’s important, it’s the pathway and the solution to reach it that are important. Typical student problem? Open the book to chapter X to apply known procedure Y to determine an unknown answer.

Shout out to Bloom’s! Here’s Eric’s slide to remind you.

Rights reside with Eric Mazur.

Rights reside with Eric Mazur.

Eric doesn’t think that many of us, including Harvard, even reach the Applying stage. He referred to a colleague in physics who used baseball problems throughout the course in assignments, until he reached the final exam where he ran out of baseball problems and used football problems. “Professor! We’ve never done football problems!” Eric noted that, while the audience were laughing, we should really be crying. If we can’t apply what we’ve learned then we haven’t actually learned i.

Eric sneakily put more audience participation into the talk with an open ended question that appeared to not have enough information to come up with a solution, as it required assumptions and modelling. From a Bloom’s perspective, this is right up the top.

Students loathe assumptions? Why? Mostly because we’ll give them bad marks if they get it wrong. But isn’t the ability to make assumptions a really important skill? Isn’t this fundamental to success?

Eric demonstrated how to tame the problem by adding in more constraints but this came at the cost of the creating stage of Bloom’s and then the evaluating and analysing(Check out his slides, pages 31 to 40, for details of this.) If you add in the memorisation of the equation, we have taken all of the guts out of the problem, dropping down to the lowest level of Bloom’s.

But, of course, computers can do most of the hard work for that is mechanistic. Problems at the bottom layer of Bloom’s are going to be solved by machines – this is not something we should train 21st Century students for.

But… real problem solving is erratic. Riddled with fuzziness. Failure prone. Not guaranteed to succeed. Most definitely not guaranteed to be optimal. The road to success is littered with failures.

But, if you make mistakes, you lose marks. But if you’re not making mistakes, you’re very unlikely to be creative and innovative and this is the problem with our assessment practices.

Eric showed us a stress of a traditional exam room: stressful, isolated, deprived of calculators and devices. Eric’s joke was that we are going to have to take exams naked to ensure we’re not wearing smart devices. We are in a time and place where we can look up whatever we want, whenever we want. But it’s how you use that information that makes a difference. Why are we testing and assessing students under such a set of conditions? Why do we imagine that the result we get here is going to be any indicator at all of the likely future success of the student with that knowledge?

Cramming for exams? Great, we store the information in short-term memory. A few days later, it’s all gone.

Assessment produces a conflict, which Eric noticed when he started teaching a team and project based course. He was coaching for most of the course, switching to a judging role for the monthly fair. He found it difficult to judge them because he had a coach/judge conflict. Why do we combine it in education when it would be unfair or unpleasant in every other area of human endeavour? We hide between the veil of objectivity and fairness. It’s not a matter of feelings.

But… we go back to Bloom’s. The only thinking skill that can be evaluated truly objectively is remembering, at the bottom again.

But let’s talk about grade inflation and cheating. Why do people cheat at education when they don’t generally cheat at learning? But educational systems often conspire to rob us of our ownership and love of learning. Our systems set up situations where students cheat in order to succeed.

  • Mimic real life in assessment practices!

Open-book exams. Information sticks when you need it and use it a lot. So use it. Produce problems that need it. Eric’s thought is you can bring anything you want except for another living person. But what about assessment on laptops? Oh no, Google access! But is that actually a problem? Any question to which the answer can be Googled is not an authentic question to determine learning!

Eric showed a video of excited students doing a statistic tests as a team-based learning activity. After an initial pass at the test, the individual response is collected (for up to 50% of the grade), and then students work as a group to confirm the questions against an IF AT scratchy card for the rest of the marks. Discussion, conversation, and the students do their own grading for you. They’ve also had the “A-ha!” moment. Assessment becomes a learning opportunity.

Eric’s not a fan of multiple choice so his Learning Catalytics software allows similar comparison of group answers without having to use multiple choice. Again, the team based activities are social, interactive and must less stressful.

  • Focus on feedback, not ranking.

Objective ranking is a myth. The amount of, and success with, advanced education is no indicator of overall success in many regards. So why do we rank? Eric showed some graphs of his students (in earlier courses) plotting final grades in physics against the conceptual understanding of force. Some people still got top grades without understanding force as it was redefined by Newton. (For those who don’t know, Aristotle was wrong on this one.) Worse still is the student who mastered the concept of force and got a C, when a student who didn’t master force got an A. Objectivity? Injustice?

  • Focus on skills, not content

Eric referred to Wiggins and McTighe, “Understanding by Design.”  Traditional approach is course content drives assessment design. Wiggins advocates identifying what the outcomes are, formulate these as action verbs, ‘doing’ x rather than ‘understanding’ x. You use this to identify what you think the acceptable evidence is for these outcomes and then you develop the instructional approach. This is totally outcomes based.

  • resolve coach/judge conflict

In his project-based course, Eric brought in external evaluators, leaving his coach role unsullied. This also validates Eric’s approach in the eyes of his colleagues. Peer- and self-evaluation are also crucial here. Reflective time to work out how you are going is easier if you can see other people’s work (even anonymously). Calibrated peer review, cpr.molsci.ucla.edu, is another approach but Eric ran out of time on this one.

If we don’t rethink assessment, the result of our assessment procedures will never actually provide vital information to the learner or us as to who might or might not be successful.

I really enjoyed this talk. I agree with just about all of this. It’s always good when an ‘internationally respected educator’ says it as then I can quote him and get traction in change-driving arguments back home. Thanks for a great talk!

 


Musing on Industrial Time

Now Print, Black, Linocut, (C) Nick Falkner, 2013

I caught up with a good friend recently and we were discussing the nature of time. She had stepped back from her job and was now spending a lot of her time with her new-born son. I have gone to working three days a week, hence have also stepped back from the five-day grind.  It was interesting to talk about how this change to our routines had changed the way that we thought of and used time. She used a term that I wanted to discuss here, which was industrial timeto describe the clock-watching time of the full-time worker. This is part of the larger area of time discipline, how our society reacts to and uses time, and is really quite interesting. Both of us had stopped worrying about the flow of time in measurable hours on certain days and we just did things until we ran out of day. This is a very different activity from the usual “do X now, do Y in 15 minutes time” that often consumes us. In my case, it took me about three months of considered thought and re-training to break the time discipline habits of thirty years. In her case, she has a small child to help her to refocus her time sense on the now.

Modern time-sense is so pervasive that we often don’t think about some of the underpinnings of our society. It is easy to understand why we have years and, although they don’t line up properly, months given that these can be matched to astronomical phenomena that have an effect on our world (seasons and tides, length of day and moonlight, to list a few). Days are simple because that’s one light/dark cycle. But why there are 52 weeks in a year? Why are there 7 days in a week? Why did the 5-day week emerge as a contiguous block of 5 days? What is so special about working 9am to 5pm?

A lot of modern time descends from the struggle of radicals and unionists to protect workers from the excesses of labour, to stop people being worked to death, and the notion of the 8 hour day is an understandable division of a 24 hour day into three even chunks for work, rest and leisure. (Goodness, I sound like I’m trying to sell you chocolate!)

If we start to look, it turns out that the 7 day week is there because it’s there, based on religion and tradition. Interestingly enough, there have been experiments with other week lengths but it appears hard to shift people who are used to a certain routine and, tellingly, making people wait longer for days off appears to be detrimental to adoption.

If we look at seasons and agriculture, then there is a time to sow, to grow, to harvest and to clear, much as there is a time for livestock to breed and to be raised for purpose. If we look to the changing time of sunrise and sunset, there is a time at which natural light is available and when it is not. But, from a time discipline perspective, these time systems are not enough to be able to build a large-scale, industrial and synchronised society upon – we must replace a distributed, loose and collective notion of what time is with one that is centralised, authoritarian and singular. While religious ceremonies linked to seasonal and astronomical events did provide time-keeping on a large scale prior to the industrial revolution, the requirement for precise time, of an accuracy to hours and minutes, was not possible and, generally, not required beyond those cues given from nature such as dawn, noon, dusk and so on.

After the industrial revolution, industries and work was further developed that was heavily separated from a natural linkage – there are no seasons for a coal mine or a steam engine – and the development of the clock and reinforcement of the calendar of work allowed both the measurement of working hours (for payment) and the determination of deadlines, given that natural forces did not have to be considered to the same degree. Steam engines are completed, they have no need to ripen.

With the notion of fixed and named hours, we can very easily determine if someone is late when we have enough tools for measuring the flow of time. But this is, very much, the notion of the time that we use in order to determine when a task must be completed, rather than taking an approach that accepts that the task will be completed at some point within a more general span of time.

We still have confusion where our understanding of “real measures” such as days, interact with time discipline. Is midnight on the 3rd of April the second after the last moment of April the 2nd or the second before the first moment of April the 4th? Is midnight 12:00pm or 12:00am? (There are well-defined answers to this but the nature of the intersection is such that definitions have to be made.)

But let’s look at teaching for a moment. One of the great criticisms of educational assessment is that we confuse timeliness, and in this case we specifically mean an adherence to meeting time discipline deadlines, with achievement. Completing the work a crucial hour after it is due can lead to that work potentially not being marked at all, or being rejected. But we do usually have over-riding reasons for doing this but, sadly, these reasons are as artificial as the deadlines we impose. Why is an Engineering Degree a four-year degree? If we changed it to six would we get better engineers? If we switched to competency based training, modular learning and life-long learning, would we get more people who were qualified or experienced with engineering? Would we get less? What would happen if we switched to a 3/1/2/1 working week? Would things be better or worse? It’s hard to evaluate because the week, and the contiguous working week, are so much a part of our world that I imagine that today is the first day that some of you have thought about it.

Back to education and, right now, we count time for our students because we have to work out bills and close off accounts at end of financial year, which means we have to meet marking and award deadlines, then we have to project our budget, which is yearly, and fit that into accredited degree structures, which have year guidelines…

But I cannot give you a sound, scientific justification for any of what I just wrote. We do all of that because we are caught up in industrial time first and we convince ourselves that building things into that makes sense. Students do have ebb and flow. Students are happier on certain days than others. Transition issues on entry to University are another indicator that students develop and mature at different rates – why are we still applying industrial time from top to bottom when everything we see here says that it’s going to cause issues?

Oh, yes, the “real world” uses it. Except that regular studies of industrial practice show that 40 hour weeks, regular days off, working from home and so on are more productive than the burn-out, everything-late, rush that we consider to be the signs of drive. (If Henry Ford thinks that making people work more than 40 hours a week is bad for business, he’s worth listening to.) And that’s before we factor in the development of machines that will replace vast numbers of human jobs in the next 20 years.

I have a different approach. Why aren’t we looking at students more like we regard our grape vines? We plan, we nurture, we develop, we test, we slowly build them to the point where they can produce great things and then we sustain them for a fruitful and long life. When you plant grape vines, you expect a first reasonable crop level in three years, and commercial levels at five. Tellingly, the investment pattern for grapes is that it takes you 10 years to break even and then you start making money back. I can’t tell you how some of my students will turn out until 15-25 years down the track and it’s insanity to think you can base retrospective funding on that timeframe.

You can’t make your grapes better by telling them to be fruitful in two years. Some vines take longer than others. You can’t even tell them when to fruit (although can trick them a little). Yet, somehow, we’ve managed to work around this to produce a local wine industry worth around $5 billion dollars. We can work with variation and seasonal issues.

One of the reasons I’m so keen on MOOCs is that these can fit in with the routines of people who can’t dedicate themselves to full-time study at the moment. By placing well-presented, pedagogically-sound materials on-line, we break through the tyranny of the 9-5, 5 day work week and let people study when they are ready to, where they are ready to, for as long as they’re ready to. Like to watch lectures at 1am, hanging upside down? Go for it – as long as you’re learning and not just running the video in the background while you do crunches, of course!

Once you start to question why we have so many days in a week, you quickly start to wonder why we get so caught up on something so artificial. The simple answer is that, much like money, we have it because we have it. Perhaps it’s time to look at our educational system to see if we can do something that would be better suited to developing really good knowledge in our students, instead of making them adept at sliding work under our noses a second before it’s due. We are developing systems and technologies that can allow us to step outside of these structures and this is, I believe, going to be better for everyone in the process.

Conformity isn’t knowledge, and conformity to time just because we’ve always done that is something we should really stop and have a look at.


That’s not the smell of success, your brain is on fire.

Would you mind putting out the hippocampus when you have a chance?

Would you mind putting out the hippocampus when you have a chance?

I’ve written before about the issues of prolonged human workload leading to ethical problems and the fact that working more than 40 hours a week on a regular basis is downright unproductive because you get less efficient and error-prone. This is not some 1968 French student revolutionary musing on what benefits the soul of a true human, this is industrial research by Henry Ford and the U.S. Army, neither of whom cold be classified as Foucault-worshipping Situationist yurt-dwelling flower children, that shows that there are limits to how long you can work in a sustained weekly pattern and get useful things done, while maintaining your awareness of the world around you.

The myth won’t die, sadly, because physical presence and hours attending work are very easy to measure, while productive outputs and their origins in a useful process on a personal or group basis are much harder to measure. A cynic might note that the people who are around when there is credit to take may end up being the people who (reluctantly, of course) take the credit. But we know that it’s rubbish. And the people who’ve confirmed this are both philosophers and the commercial sector. One day, perhaps.

But anyone who has studied cognitive load issues, the way that the human thinking processes perform as they work and are stressed, will be aware that we have a finite amount of working memory. We can really only track so many things at one time and when we exceed that, we get issues like the helmet fire that I refer to in the first linked piece, where you can’t perform any task efficiently and you lose track of where you are.

So what about multi-tasking?

Ready for this?

We don’t.

There’s a ton of research on this but I’m going to link you to a recent article by Daniel Levitin in the Guardian Q&A. The article covers the fact that what we are really doing is switching quickly from one task to another, dumping one set of information from working memory and loading in another, which of course means that working on two things at once is less efficient than doing two things one after the other.

But it’s more poisonous than that. The sensation of multi-tasking is actually quite rewarding as we get a regular burst of the “oooh, shiny” rewards our brain gives us for finding something new and we enter a heightened state of task readiness (fight or flight) that also can make us feel, for want of a better word, more alive. But we’re burning up the brain’s fuel at a fearsome rate to be less efficient so we’re going to tire more quickly.

Get the idea? Multi-tasking is horribly inefficient task switching that feels good but makes us tired faster and does things less well. But when we achieve tiny tasks in this death spiral of activity, like replying to an e-mail, we get a burst of reward hormones. So if your multi-tasking includes something like checking e-mails when they come in, you’re going to get more and more distracted by that, to the detriment of every other task. But you’re going to keep doing them because multi-tasking.

I regularly get told, by parents, that their children are able to multi-task really well. They can do X, watch TV, do Y and it’s amazing. Well, your children are my students and everything I’ve seen confirms what the research tells me – no, they can’t but they can give a convincing impression when asked. When you dig into what gets produced, it’s a different story. If someone sits down and does the work as a single task, it will take them a shorter time and they will do a better job than if they juggle five things. The five things will take more than five times as long (up to 10, which really blows out time estimation) and will not be done as well, nor will the students learn about the work in the right way. (You can actually sabotage long term storage by multi-tasking in the wrong way.) The most successful study groups around the Uni are small, focused groups that stay on one task until it’s done and then move on. The ones with music and no focus will be sitting there for hours after the others are gone. Fun? Yes. Efficient? No. And most of my students need to be at least reasonably efficient to get everything done. Have some fun but try to get all the work done too – it’s educational, I hear. 🙂

It’s really not a surprise that we haven’t changed humanity in one or two generations. Our brains are just not built in a way that can (yet) provide assistance with the quite large amount of work required to perform multi-tasking.

We can handle multiple tasks, no doubt at all, but we’ve just got to make sure, for our own well-being and overall ability to complete the task, that we don’t fall into the attractive, but deceptive, trap that we are some sort of parallel supercomputer.


Rules: As For Them, So For Us

In a previous post, I mentioned a game called “Dog Eat Dog” where players role-play the conflict between Colonist and Native Occupiers, through playing out scenarios that both sides seek to control, with the result being the production of a new rule that encapsulates the ‘lesson’ of the scenario. I then presented education as being a good fit for this model but noted that many of the rules that students have to be obey are behavioural rather than knowledge-focussed. A student who is ‘playing through’ education will probably accumulate a list of rules like this (not in any particular order):

  1. Always be on time for class
  2. Always present your own work
  3. Be knowledgable
  4. Prepare for each activity
  5. Participate in class
  6. Submit your work on time

But, as noted in Dog Eat Dog, the nasty truth of colonisation is that the Colonists are always superior to the Colonised. So, rule 0 is actually: Students are inferior to Teachers. Now, that’s a big claim to make – that the underlying notion in education is one of inferiority. In the Dog Eat Dog framing, the superiority manifests as dominance in decision making and the ability to intrude into every situation. We’ll come back to this.

If we tease apart the rules for students then are some obvious omissions that we would like to see such as “be innovative” or “be creative”, except that these rules are very hard to apply as pre-requisites for progress. We have enough potential difficulty with the measurement of professional skills, without trying to assess if one thing is a creative approach while another is just missing the point or deliberate obfuscation. It’s understandable that five of the rules presented are those that we can easily control with extrinsic motivational factors – 1, 2, 4, 5, and 6 are generally presented as important because of things like mandatory attendance, plagiarism rules and lateness penalties. 3, the only truly cognitive element on the list, is a much harder thing to demand and, unsurprisingly, this is why it’s sometimes easier to seek well-behaved students than it is to seek knowledgable, less-controlled students, because it’s so much harder to see that we’ve had a  positive impact. So, let us accept that this list is naturally difficult to select and somewhat artificial, but it is a reasonable model for what people expect of a ‘good’ student.

Let me ask you some questions before we proceed.

  1. A student is always late for class. Could there be a reasonable excuse for this and, if so, does your system allow for it?
  2. Students occasionally present summary presentations from other authors, including slides prepared by scholarly authors. How do you interpret that?
  3. Students sometimes show up for classes and are obviously out of their depth. What do you do? Should they go away and come back later when they’re ready? Do they just need to try harder?
  4. Students don’t do the pre-reading and try to cram it in just before a session. Is this kind of “just in time” acceptable?
  5. Students sometimes sit up the back, checking their e-mail, and don’t really want to get involved. Is that ok? What if they do it every time?
  6. Students are doing a lot of things and often want to shift around deadlines or get you to take into account their work from other courses or outside jobs. Do you allow this? How often? Is there a penalty?

As you can see, I’ve taken each of the original ‘good student’ points and asked you to think about it. Now, let us accept that there are ultimate administrative deadlines (I’ve already talked about this a lot in time banking) and we can accept that the student is aware of these and are not planning to put all their work off until next century.

Now, let’s look at this as it applies to teaching staff. I think we can all agree that a staff member who meets that list are going to achieve a lot of their teaching goals. I’m going to reframe the questions in terms of staff.

  1. You have to drop your kids off every morning at day care. This means that you show up at your 9am lecture 5 minutes late every day because you physically can’t get there any faster and your partner can’t do it because he/she is working shift work. How do you explain this to your students?
  2. You are teaching a course from a textbook which has slides prepared already. Is it ok to take these slides and use them without any major modification?
  3. You’ve been asked to cover another teacher’s courses for two weeks due to their illness. You have a basis in the area but you haven’t had to do anything detailed for it in over 10 years and you’ll also have to give feedback on the final stages of a lengthy assignment. How do you prepare for this and what, if anything, do you tell the class to brief them on your own lack of expertise?
  4. The staff meeting is coming around and the Head of School wants feedback on a major proposal and discussion at that meeting. You’ve been flat out and haven’t had a chance to look at it, so you skim it on the way to the meeting and take it with you to read in the preliminaries. Given the importance of the proposal, do you think this is a useful approach?
  5. It’s the same staff meeting and Doctor X is going on (again) about radical pedagogy and Situationist philosophy. You quickly catch up on some important work e-mails and make some meetings for later in the week, while you have a second.
  6. You’ve got three research papers due, a government grant application and your Head of School needs your workload bid for the next calendar year. The grant deadline is fixed and you’ve already been late for three things for the Head of School. Do you drop one (or more) of the papers or do you write to the convenors to see if you can arrange an extension to the deadline?

Is this artificial? Well, of course, because I’m trying to make a point. Beyond being pedantic on this because you know what I’m saying, if you answered one way for the staff member and other way for the student then you have given the staff member more power in the same situation than the student. Just because we can all sympathise with the staff member (Doctor X sounds horribly familiar, doesn’t he?) doesn’t that the student’s reasons, when explored and contextualised, are not equally valid.

If we are prepared to listen to our students and give their thoughts, reasoning and lives as much weight and value as our own, then rule 0 is most likely not in play at the moment – you don’t think your students are inferior to you. If you thought that the staff member was being perfectly reasonable and yet you couldn’t see why a student should be extended the same privileges, even where I’ve asked you to consider the circumstances where it could be, then it’s possible that the superiority issue is one that has become well-established at your institution.

Ultimately, if this small list is a set of goals, then we should be a reasonable exemplar for our students. Recently, due to illness, I’ve gone from being very reliable in these areas, to being less reliable on things like the level of preparation I used to do and timeliness. I have looked at what I’ve had to do and renegotiated my  deadlines, apologising and explaining where I need to. As a result, things are getting done and, as far as I know, most people are happy with what I’m doing. (That’s acceptable but they used to be very happy. I have  way to go.) I still have a couple of things to fix, which I haven’t forgotten about, but I’ve had to carry out some triage. I’m honest about this because, that way, I encourage my students to be honest with me. I do what I can, within sound pedagogical framing and our administrative requirements, and my students know that. It makes them think more, become more autonomous and be ready to go out and practice at a higher level, sooner.

This list is quite deliberately constructed but I hope that, within this framework, I’ve made my point: we have to be honest if we are seeing ourselves as superior and, in my opinion, we should work more as equals with each other.


Tukhta: the tyranny of inflated performance figures.

I’m sketching out a book on the early Soviet Union and artistic movements (don’t ask) so I’ve been rereading every Russian author I can get my hands on. I read a lot of these works when I was (probably too) young, starting from the very easy and shallow slopes of “Ivan Denisovich” and then plunging down into “Gulag Archipelago”. One of the things that comes out starkly from Solzhenitsyn’s account of the forced labour camps of “Gulag Archipelago” is the way that unrealistic expectations from an overbearing superior organisation can easily lead to an artificial conformity to productivity requirements, which leads to people cheating to achieve their overly ambitious quotas. In Solzhenitsyn’s words, the many thieves in the camp (he is less than complementary about non-political prisoners) coined the word tufta, which he rendered into better Russian as tukhta, the practice of making up your quotas through devious means and fabricating outputs. This could be as simple as writing down a figure that didn’t reflect your actual labour or picking up a pile of timber that had already been counted, moving it somewhere else, and counting it again.

The biggest problem with achieving a unreasonable goal, especially one which is defined by ideology rather than reality, is that it is easy for those who can to raise the expectation because, if you can achieve that goal, then no doubt you can achieve this one. This led to such excesses as the Stakhanovite movement, where patently impossible levels of human endeavour were achieved as evidence of commitment to Stalinist ideology and being a good member of the state. The darker side to all this, and this will be a word very familiar to those used to Soviet history, is that anyone who doesn’t attain such lofty goals or doesn’t sign up to be a noble Stakhanovite is labelled as a wrecker. Wreckers were a very common obstacle in the early development of the new Soviet state, pointing out things like “you can’t build that without concrete” or “water flows downhill”.  It should be noted that the original directives of the movement were quite noble, as represented in this extract from a conference in 1935:

The Stakhanovite movement means organizing labor in a new fashion, rationalizing technologic processes, correct division of labor, liberating qualified workers from secondary spadework, improving work place, providing rapid growth for labor productivity and securing significant increase of workers’ salaries.

Pretty good, right? Now consider that the namer of this movement was “Aleksei Stakhanov, who had mined 102 tons of coal in less than 6 hours (14 times his quota)”. This astounding feat of human endeavour was broken a year later, when Nikita Izotov mined 607 tons of coal in a single shift! It’s worth noting that fully-mechanised and highly industrialised contemporary Australian coal mines can produce round about 3,800 tonnes every 6 hours. What a paltry achievement when all you need is six Nikita Izotovs. So this seemingly well-focused initiative, structured as a benefit to state and worker, is disingenuous for the state and dangerous for the worker.

""Stakhanovite model soviet worker guarantees the continuing peace!"" You'll note the anti-intelligensia and racist targets of the worker - ideologically these were all wreckers.

“”Stakhanovite model soviet worker guarantees the continuing peace!””
You’ll note the anti-intelligensia and racist imagery on the poster as well – ideologically these were all wreckers.

Imagine that you are a worker trying to keep yourself and your family alive in the middle of famine after famine – of course you want to meet the requirements as well as you can, potentially even exceeding them so that you don’t get sent to a camp, locked up, or demoted and diminished in your role. While some people might be practising tukhta out of laziness, you are practising it because it is the way that things are. You need to nod in agreement with ridiculous requirements and then write up your results in a way that exceeds them, if you want to survive. Your reward? Even more ridiculous requirements, not determined in capacity and available inputs but in required output. Tukhta is your curse and your only means of survival. Unsurprisingly, the Stakhanovite movement was denounced as part of Stalinism later on in the emerging and mutating Soviet Union.

Now imagine that you are a student. You have been given a pile of reading to do, a large collection of assignments across a variety of subjects that are not really linked to each other, and you are told that you need to do all of this to succeed. Are you going to deeply apply yourself to everything, to form your own conceptual framework and illuminate it through careful study? Well, perhaps you would, except that you have quotas to achieve and deadlines to meet and, around you, other students are doing better, pressing further and are being actively rewarded and encouraged for it. Will you be at least tempted to move things around to achieve your quota? Will you prioritise some labour over another, which could be more useful in the long-term? Will you hide your questions in the hope of being able to be seen to not be a bad student?

Now imagine that you are a young academic, perhaps one with a young family, and you are going to enter the job market. You know that your publications, research funding and overall contributions will be compared to other stand-outs in the field, to overall averages and to defined requirements for the institution. Will you sit and mull contemplatively over an important point of science or will you crank out yet another journal at a prestigious, but not overly useful, target venue, working into the night and across the weekend? Will you look at the exalted “Research Stars” who have very high publication and citation rates and who attract salary loadings up to a level that could pay for 2-3 times the number of positions they hold? Will you be compared to these people and found wanting? Will you write papers with anyone prestigious? Will you do what you need to do to move from promising to reliable to a leader in the field regardless of whether it’s actually something you should be doing? (Do you secretly wonder whether you can even get there from where you started and lie awake at night thinking about it?)

Measurements that pit us against almost impossible standards and stars so high that we probably cannot reach them grind down the souls of the majority of the population and lead them into the dark pathways of tukhta. It is easy to say “Don’t cheat” or “Don’t work all weekend” when you are on top of the pile. As the workers in the Gulag and many Soviet Citizens found out, doing that just lets the people setting the quotas to keep setting them as they wish, with no concern for the people who are grist to the mill.

Tukhta should not be part of an educational system and we should be very wary of the creeping mensuration of the academy. You don’t have to look far to see highly celebrated academics and researchers who were detected in their cheating and were punished hard. Yet a part of me knows that the averages are set as much by the tukhtaviks that we have not yet detected and, given how comparative was have made our systems, that is monstrously unfair.

Assessing how well someone is performing needs to move beyond systems that are so pitifully easy to game and so terribly awful to their victims when they are so gamed.