Do any of you remember that scene from “Pretty Woman”, when Richard Gere’s character Edward asks his ex-girlfriend Susan if she had spent more time talking to his secretary than to him? Her reply was, simply, “she was one of my bridesmaids”. Any time in a relationship of any kind that you don’t meet the needs of the people in the relationship, it’s going to cause problems. (Not all problems have to end with you scaling a ladder to patch up your relationship with Julia Roberts but there will be problems.) Now, before anyone is wondering (or my friends are worried), my marriage is still great, I’m talking about my professional relationship with my students.
While I’ve been critical of myself and my teaching this semester, where I’ve done a good job but not necessarily excelled across the board, I haven’t identified one of the greatest problems that has crept in – no-one in their right mind should be emulating what I laughingly refer to as my work ethic, time commitment or current pursuit of success. Right now, I am not a good role model for students. While I am still ethical, professional, knowledgable and I’m apparently doing a good job, I cannot present myself as a role model to my students because I am losing the time that I need to seize new opportunities, and to allow for the unrushed catch-up time with other people that is vital to doing a job such as mine and doing it well. At the moment, any student who comes to see me does so knowing that I will have 15-30 minutes, tops, and that I will then have to rush off, elsewhere, to go and do something else. It doesn’t matter when they come to see me – if I’m in at 7:30am it’s because the day is starting at 8. If I’m still there at 7pm at night it’s because I have to be in order to meet requirements for that day or the next. Let me give you an example from my lectures.
My Student Experience of Learning and Teaching results have come in and there have been a lot of warm and rewarding comments from my students, among a pleasing overall rating. But one of my students hit the nail on the head. “[Nick] always seems to have a lot to say and constantly looks at his watch. (I assume that it’s to keep within time constraints) the problem is that he feels like he’s rushing.”
Ouch. That’s far too true and, while only one student noted it, you bet every other student was watching me use my watch to check my time progress through a busy, informative but ultimately time constrained lecture and at least some of them thought “Hmm, I have a question but I don’t want to bother him.”
It’s my job to be bothered! It’s my job to answer questions! Right now, it’s pretty obvious that students are getting the vibe that I’m a good lecturer, I care about them, I’m working well to give them the right knowledge and they love the course that I built… but… they don’t want to bother me because I’m too busy. Because I look too busy.
Every student who comes to see does so in the one of the windows that I have in my day, often between meetings, my meetings back up on each other with monotonous regularity and, looking at my calendar for last, week, the total amount of time that was uncommitted prior to the week starting was…
Including the fact that Tuesday started at 7:30am and went until 6:15pm.
Please believe me when I say that I’m not boasting – I’m not proud of this, I think it’s the sign of poor scheduling and workaholism. This should be read as what it is, a sign that I have let my responsibilities pile up in a way that means that I am running the risk of becoming a stereotypically “grumpy old Professor”, who is too busy to see students.
So when you read all of this stuff about Time Banking and think “Well, I guess I can see some of his point – for assignments…” I’m trying to work out how I can take the primary goal of time banking – to make people think about their time commitments in a way that allows them to approximate a manageable uniformity of effort across time to achieve good results – and to work out how I can think about my own time in the same way. How do I adjust my boundaries in time and renegotiate while providing my own oracle and incentives for change? If I can crack that, then solving the student problem should have been made much easier.
How do I become the kind of person that I would want my students to be? Right now, it requires me to think about my commitments and my time, to treat my time as a scarce and precious commodity, but in a way that allows me to do all of the things I need to do in my job and all of the things that I love to do in my job, yet still have the time to sit around, grab a slow coffee, make a lunch booking with someone with less than a month’s notice and to get my breathing room back.
I have one of the best jobs in the world but the way I’m doing it is probably unsustainable and it’s not really in the spirit of the job that I want to do. It’s more than just me, too. I need to be seen to be approachable and to do that I have to actually be approachable, which means finding a way that makes me worthy of being a good overall role model again.
I’ve been talking about why late penalties are not only not useful but they don’t work, yet I keep talking about getting work in on time and tying it to realistic resource allocation. Does this mean I’m really using late penalties?
No, but let me explain why, starting from the underlying principle of fairness that is an aesthetic pillar of good education. One part of this is that the actions of one student should not unduly affect the learning journey of another student. That includes evaluation (and associated marks).
This is the same principle that makes me reject curve grading. It makes no sense to me that someone else’s work is judged in the context of another, when we have so little real information with which we could establish any form of equivalence of human experience and available capacity.
I don’t want to create a market economy for knowledge, where we devaluate successful demonstrations of knowledge and skill for reasons that have nothing to do with learning. Curve grading devalues knowledge. Time penalties devalue knowledge.
I do have to deal with resource constraints, in that I often have (some) deadlines that are administrative necessities, such as degree awards and things like this. I have limited human resources, both personally and professionally.
Given that I do not have unconstrained resources, the fairness principle naturally extends to say that individual students should not consume resources to the detriment of others. I know that I have a limited amount of human evaluation time, therefore I have to treat this as a constrained resource. My E1 and E2 evaluations resources must be, to a degree at least, protected to ensure the best outcome for the most students. (We can factor equity into this, and should, but this stops this from being a simple linear equivalence and makes the terms more complex than they need to be for explanation, so I’ll continue this discussion as if we’re discussing equality.)
You’ve noticed that the E3 and E4 evaluation systems are pretty much always available to students. That’s deliberate. If we can automate something, we can scale it. No student is depriving another of timely evaluation and so there’s no limitation of access to E3 and E4, unless it’s too late for it to be of use.
If we ask students to get their work in at time X, it should be on the expectation that we are ready to leap into action at second X+(prep time), or that the students should be engaged in some other worthwhile activity from X+1, because otherwise we have made up a nonsense figure. In order to be fair, we should release all of our evaluations back at the same time, to avoid accidental advantages because of the order in which things were marked. (We may wish to vary this for time banking but we’ll come back to this later.) As many things are marked in surname or student number order, the only way to ensure that we don’t accidentally keep granting an advantage is to release everything at the same time.
Remember, our whole scheme is predicated on the assumption that we have designed and planned for how long it will take to go through the work and provide feedback in time for modification before another submission. When X+(prep time) comes, we should know, roughly to the hour or day, at worst, when this will be done.
If a student hands up fifteen minutes late, they have most likely missed the preparation phase. If we delay our process to include this student, then we will delay feedback to everyone. Here is a genuine motivation for students to submit on time: they will receive rich and detailed feedback as soon as it is ready. Students who hand up late will be assessed in the next round.
That’s how the real world actually works. No-one gives you half marks for something that you do a day late. It’s either accepted or not and, often, you go to the back of the queue. When you miss the bus, you don’t get 50% of the bus. You just have to wait for the next opportunity and, most of the time, there is another bus. Being late once rarely leaves you stranded without apparent hope – unlucky Martian visitors aside.
But there’s more to this. When we have finished with the first group, we can immediately release detailed feedback on what we were expecting to see, providing the best results to students and, from that point on, anyone who submits would have the benefit of information that the first group didn’t have before their initial submission. Rather than make the first group think that they should have waited (and we know students do), we give them the best possible outcome for organising their time.
The next submission deadline is done by everyone with the knowledge gained from the first pass but people who didn’t contribute to it can’t immediately use it for their own benefit. So there’s no free-riding.
There is, of course, a tricky period between the submission deadline and the release, where we could say “Well, they didn’t see the feedback” and accept the work but that’s when we think about the message we want to send. We would prefer students to improve their time management and one part of this is to have genuine outcomes from necessary deadlines.
If we let students keep handing in later and later, we will eventually end up having these late submissions running into our requirement to give feedback. But, more importantly, we will say “You shouldn’t have bothered” to those students who did hand up on time. When you say something like this, students will learn and they will change their behaviour. We should never reinforce behaviour that is the opposite of what we consider to be valuable.
Fairness is a core aesthetic of education. Authentic time management needs to reflect the reality of lost opportunity, rather than diminished recognition of good work in some numerical reduction. Our beauty argument is clear: we can be firm on certain deadlines and remove certain tasks from consideration and it will be a better approach and be more likely to have positive outcomes than an arbitrary reduction scheme already in use.
Before I lay out the program design I’m thinking of (and, beyond any discussion of competency, as a number of you have suggested, we are heading towards Bloom’s mastery learning as a frame with active learning elements), we need to address one of the most problematic areas of assessment.
Well, let’s be accurate, penalties are, by definition, punishments imposed for breaking the rules, so these are punishments. This is the stick in the carrot-and-stick reward/punish approach to forcing people to do what you want.
Let’s throw the Greek trinity at this and see how it shapes up. A student produces an otherwise perfect piece of work for an assessment task. It’s her own work. She has spent time developing it. It’s really good. Insightful. Oh, but she handed it up a day late. So we’re now going to say that this knowledge is worth less because it wasn’t delivered on time. She’s working a day job to pay the bills? She should have organised herself better. No Internet at home? Why didn’t she work in the library? I’m sure the campus is totally safe after hours and, well, she should just be careful in getting to and from the library. After all, the most important thing in her life, without knowing anything about her, should be this one hundred line program to reinvent something that has been written over a million times by every other CS student in history.
That’s not truth. That’s establishing a market value for knowledge with a temporal currency. To me, unless there’s a good reason for doing this, this is as bad as curve grading because it changes what the student has achieved for reasons outside of the assignment activity itself.
“Ah!” you say “Nick, we want to teach people to hand work in on time because that’s how the world works! Time is money, Jones!”
Rubbish. Yes, there are a (small) number of unmovable deadlines in the world. We certainly have some in education because we have to get grades in to achieve graduations and degrees. But most adults function in a world where they choose how to handle all of the commitments in their lives and then they schedule them accordingly. The more you do that, the more practice you get and you can learn how to do it well.
If you have ever given students a week, or even a day’s, extension because of something that has stopped you being able to accept or mark student work, no matter how good the reason, you have accepted that your submission points are arbitrary. (I feel strongly about this and have posted about it before.)
So what would be a good reason for sticking to these arbitrary deadlines? We’d want to see something really positive coming out of the research into this, right? Let’s look at some research on this, starting with Britton and Tesser, “Effects of Time-Management Practices on College Grades”, J Edu Psych, 1991, 83, 3. This reinforces what we already know from Bandura: students who feel in control and have high self-efficacy are going to do well. If a student sits down every day to work out what they’re going to do then they, unsurprisingly, can get things done. But this study doesn’t tell us about long-range time planning – the realm of instrumentality, the capability to link activity today with success in the future. (Here are some of my earlier thoughts on this, with references to Husman.) From Husman, we know that students value tasks in terms of how important they think it is, how motivated they are and how well they can link future success to the current task.
In another J Edu Psych paper (1990,82,4), Macan and Shahani reported that participants who felt that they had control over what they were doing did better but also clearly indicated that ambiguity and stress had an influence on time management in terms of perception and actuality. But the Perceived Control of Time (author’s caps) dominated everything, reducing the impact of ambiguity, reducing the impact of stress, and lead to greater satisfaction.
Students are rarely in control of their submission deadlines. Worse, we often do not take into account everything else in a student’s life (even other University courses) when we set our own deadlines. Our deadlines look arbitrary to students because they are, in the majority of cases. There’s your truth. We choose deadlines that work for our ability to mark and to get grades in or, perhaps, based on whether we are in the country or off presenting research on the best way to get students to hand work in on-time.
(Yes, the owl above is staring at me just as hard as he is staring at anyone else here.)
My own research clearly shows that fixed deadlines do not magically teach students the ability to manage their time and, when you examine it, why should it? (ICER 2012, was part of a larger study that clearly demonstrated students continuing, and even extending, last-minute behaviour all the way to the fourth year of their studies.) Time management is a discipline that involves awareness of the tasks to be performed, a decomposition of those tasks to subtasks that can be performed when the hyperbolic time discounting triggers go off, and a well-developed sense of instrumentality. Telling someone to hand in their work by this date OR ELSE does not increase awareness, train decomposition, or develop any form of planning skills. Well, no wonder it doesn’t work any better than shouting at people teaches them Maxwell’s Equations or caning children suddenly reveals the magic of the pluperfect form in Latin grammar.
So, let’s summarise: students do well when they feel in control and it helps with all of the other factors that could get in the way. So, in order to do almost exactly the opposite of help with this essential support step, we impose frequently arbitrary time deadlines and then act surprised when students fall prey to lack of self-confidence, stress or lose sight of what they’re trying to do. They panic, asking lots of (what appear to be) unnecessary questions because they are desperately trying to reduce confusion and stress. Sound familiar?
I have written about this at length while exploring time banking, giving students agency and the ability to plan their own time, to address all of these points. But the new lens in my educational inspection loupe allows me to be very clear about what is most terribly wrong with late penalties.
They are not just wrong, they satisfy none of anyone’s educational aesthetics. Because we don’t take a student’s real life into account, we are not being fair. Because we are not actually developing the time management abilities but treating them as something that will be auto-didactically generated, we are not being supportive. Because we downgrade work when it is still good, we are being intellectually dishonest. Because we vary deadlines to suit ourselves but may not do so for an individual student, we are being hypocritical. We are degrading the value of knowledge for procedural correctness. This is hideously “unbeautiful”.
That is not education. That’s bureaucracy. Just because most of us live within a bureaucracy doesn’t mean that we have to compromise our pedagogical principles. Even trying to make things fit well, as Rapaport did to try and fit into another scale, we end up warping and twisting our intent, even before we start thinking about lateness and difficult areas such as that. This cannot be good.
There is nothing to stop a teacher setting an exercise that is about time management and is constructed so that all steps will lead someone to develop better time management. Feedback or marks that reflect something being late when that is the only measure of fitness is totally reasonable. But to pretend that you can slap some penalties on to the side of an assessment and it will magically self-scaffold is to deceive yourself, to your students’ detriment. It’s not true.
Do I have thoughts on how to balance marking resources with student feedback requirements, elastic time management, and real assessments while still recognising that there are some fixed deadlines?
Funny you should ask. We’ll come back to this, soon.
In a previous post, I mentioned a game called “Dog Eat Dog” where players role-play the conflict between Colonist and Native Occupiers, through playing out scenarios that both sides seek to control, with the result being the production of a new rule that encapsulates the ‘lesson’ of the scenario. I then presented education as being a good fit for this model but noted that many of the rules that students have to be obey are behavioural rather than knowledge-focussed. A student who is ‘playing through’ education will probably accumulate a list of rules like this (not in any particular order):
- Always be on time for class
- Always present your own work
- Be knowledgable
- Prepare for each activity
- Participate in class
- Submit your work on time
But, as noted in Dog Eat Dog, the nasty truth of colonisation is that the Colonists are always superior to the Colonised. So, rule 0 is actually: Students are inferior to Teachers. Now, that’s a big claim to make – that the underlying notion in education is one of inferiority. In the Dog Eat Dog framing, the superiority manifests as dominance in decision making and the ability to intrude into every situation. We’ll come back to this.
If we tease apart the rules for students then are some obvious omissions that we would like to see such as “be innovative” or “be creative”, except that these rules are very hard to apply as pre-requisites for progress. We have enough potential difficulty with the measurement of professional skills, without trying to assess if one thing is a creative approach while another is just missing the point or deliberate obfuscation. It’s understandable that five of the rules presented are those that we can easily control with extrinsic motivational factors – 1, 2, 4, 5, and 6 are generally presented as important because of things like mandatory attendance, plagiarism rules and lateness penalties. 3, the only truly cognitive element on the list, is a much harder thing to demand and, unsurprisingly, this is why it’s sometimes easier to seek well-behaved students than it is to seek knowledgable, less-controlled students, because it’s so much harder to see that we’ve had a positive impact. So, let us accept that this list is naturally difficult to select and somewhat artificial, but it is a reasonable model for what people expect of a ‘good’ student.
Let me ask you some questions before we proceed.
- A student is always late for class. Could there be a reasonable excuse for this and, if so, does your system allow for it?
- Students occasionally present summary presentations from other authors, including slides prepared by scholarly authors. How do you interpret that?
- Students sometimes show up for classes and are obviously out of their depth. What do you do? Should they go away and come back later when they’re ready? Do they just need to try harder?
- Students don’t do the pre-reading and try to cram it in just before a session. Is this kind of “just in time” acceptable?
- Students sometimes sit up the back, checking their e-mail, and don’t really want to get involved. Is that ok? What if they do it every time?
- Students are doing a lot of things and often want to shift around deadlines or get you to take into account their work from other courses or outside jobs. Do you allow this? How often? Is there a penalty?
As you can see, I’ve taken each of the original ‘good student’ points and asked you to think about it. Now, let us accept that there are ultimate administrative deadlines (I’ve already talked about this a lot in time banking) and we can accept that the student is aware of these and are not planning to put all their work off until next century.
Now, let’s look at this as it applies to teaching staff. I think we can all agree that a staff member who meets that list are going to achieve a lot of their teaching goals. I’m going to reframe the questions in terms of staff.
- You have to drop your kids off every morning at day care. This means that you show up at your 9am lecture 5 minutes late every day because you physically can’t get there any faster and your partner can’t do it because he/she is working shift work. How do you explain this to your students?
- You are teaching a course from a textbook which has slides prepared already. Is it ok to take these slides and use them without any major modification?
- You’ve been asked to cover another teacher’s courses for two weeks due to their illness. You have a basis in the area but you haven’t had to do anything detailed for it in over 10 years and you’ll also have to give feedback on the final stages of a lengthy assignment. How do you prepare for this and what, if anything, do you tell the class to brief them on your own lack of expertise?
- The staff meeting is coming around and the Head of School wants feedback on a major proposal and discussion at that meeting. You’ve been flat out and haven’t had a chance to look at it, so you skim it on the way to the meeting and take it with you to read in the preliminaries. Given the importance of the proposal, do you think this is a useful approach?
- It’s the same staff meeting and Doctor X is going on (again) about radical pedagogy and Situationist philosophy. You quickly catch up on some important work e-mails and make some meetings for later in the week, while you have a second.
- You’ve got three research papers due, a government grant application and your Head of School needs your workload bid for the next calendar year. The grant deadline is fixed and you’ve already been late for three things for the Head of School. Do you drop one (or more) of the papers or do you write to the convenors to see if you can arrange an extension to the deadline?
Is this artificial? Well, of course, because I’m trying to make a point. Beyond being pedantic on this because you know what I’m saying, if you answered one way for the staff member and other way for the student then you have given the staff member more power in the same situation than the student. Just because we can all sympathise with the staff member (Doctor X sounds horribly familiar, doesn’t he?) doesn’t that the student’s reasons, when explored and contextualised, are not equally valid.
If we are prepared to listen to our students and give their thoughts, reasoning and lives as much weight and value as our own, then rule 0 is most likely not in play at the moment – you don’t think your students are inferior to you. If you thought that the staff member was being perfectly reasonable and yet you couldn’t see why a student should be extended the same privileges, even where I’ve asked you to consider the circumstances where it could be, then it’s possible that the superiority issue is one that has become well-established at your institution.
Ultimately, if this small list is a set of goals, then we should be a reasonable exemplar for our students. Recently, due to illness, I’ve gone from being very reliable in these areas, to being less reliable on things like the level of preparation I used to do and timeliness. I have looked at what I’ve had to do and renegotiated my deadlines, apologising and explaining where I need to. As a result, things are getting done and, as far as I know, most people are happy with what I’m doing. (That’s acceptable but they used to be very happy. I have way to go.) I still have a couple of things to fix, which I haven’t forgotten about, but I’ve had to carry out some triage. I’m honest about this because, that way, I encourage my students to be honest with me. I do what I can, within sound pedagogical framing and our administrative requirements, and my students know that. It makes them think more, become more autonomous and be ready to go out and practice at a higher level, sooner.
This list is quite deliberately constructed but I hope that, within this framework, I’ve made my point: we have to be honest if we are seeing ourselves as superior and, in my opinion, we should work more as equals with each other.
Currently still under a big cloud at the moment but I’m still teaching at Singapore on the weekend so I’m typing this at the airport. All of my careful plans to have items in the queue have been undermined by having a long enough protracted spell of illness (to be precise, I’m working at about half speed due to migraine or migraine-level painkillers). I have very good parts of the day where I teach and carry out all of the face-to-face things I need to do, but it drains me terribly and leaves me with no ‘extra’ time and it was the extra time I was using to do this. I’m confident that I will teach well over this weekend, I wouldn’t be going otherwise, but it will be a blur in the hotel room outside of those teaching hours.
This brings me back to the subject of deadlines. I’ve now been talking about my time banking and elastic time management ideas to a lot of people and I’ve got quite polished in my responses to the same set of questions. Let me distill them for you, as they have relevance to where I am at the moment:
- Not all deadlines can be made flexible.
I completely agree. We have to grant degrees, finalise resource allocations and so on. Banking time is about teaching time management and the deadline is the obvious focal point, but some deadlines cannot be missed. This leads me to…
- We have deadlines in industry that are fixed! Immutable! Miss it and you miss out! Why should I grant students flexible deadlines?
Because not all of your deadlines are immutable, in the same way that not all are flexible. The serious high-level government grants? The once in a lifetime opportunities to sell product X to company YYPL? Yes, they’re fixed. But to meet these fixed deadlines, we move those other deadlines that we can. We shift off other things. We work weekends. We stay up late. We delay reading something. When we learn how to manage our deadlines so that we can make time for those that are both important and immovable, we do so by managing our resources to shift other deadlines around.
Elastic time management recognises that life is full of management decisions, not mindless compliance. Pretending that some tiny assignment of pre-packaged questions we’ve been using for 10 years is the most important thing in an 18 year old’s life is not really very honest. But we do know that the students will do things if they are important and we provide enough information that they realise this!
I have had to shift a lot of deadlines to make sure that I am ready to teach for this weekend. On top of that I’ve been writing a paper that is due on the 17th of November, as well as working on many other things. How did I manage this? I quickly looked across my existing resources (and remember I’m at half-speed, so I’ve had to schedule half my usual load) and broke things down into: things that had to happen before this teaching trip, and things that could happen after. I then looked at the first list and did some serious re-arrangement. Let’s look at some of these individually.
Blog posts, which are usually prepared 1-2 days in advance, are now written on the day. My commitment to my blog is important. I think it is valuable but, and this is key, no-one else depends upon it. The blog is now allocated after everything else, which is why I had my lunch before writing this. I will still meet my requirement to post every day but it may show up some hours after my usual slot.
I haven’t been sleeping enough, which is one of the reasons that I’m in such a bad way at the moment. All of my deadlines now have to work around me getting into bed by 10pm and not getting out before 6:15am. I cannot lose any more efficiency so I have to commit serious time to rest. I have also built in some sitting around time to make sure that I’m getting some mental relaxation.
I’ve cut down my meeting allocations to 30 minutes, where possible, and combined them where I can. I’ve said ‘no’ to some meetings to allow me time to do the important ones.
I’ve pushed off certain organisational problems by doing a small amount now and then handing them to someone to look after while I’m in Singapore. I’ve sketched out key plans that I need to look at and started discussions that will carry on over the next few days but show progress is being made.
I’ve printed out some key reading for plane trips, hotel sitting and the waiting time in airports.
Finally, I’ve allocated a lot of time to get ready for teaching and I have an entire day of focus, testing and preparation on top of all of the other preparation I’ve done.
What has happened to all of the deadlines in my life? Those that couldn’t be moved, or shouldn’t be moved, have stayed where they are and the rest have all been shifted around, with the active involvement of other participants, to allow me room to do this. That is what happens in the world. Very few people have a world that is all fixed deadline and, if they do, it’s often at the expense of the invisible deadlines in their family space and real life.
I did not learn how to do this by somebody insisting that everything was equally important and that all of their work requirements trumped my life. I am learning to manage my time maturely by thinking about my time as a whole, by thinking about all of my commitments and then working out how to do it all, and to do it well. I think it’s fair to say that I learned nothing about time management from the way that my assignments were given to me but I did learn a great deal from people who talked to me about their processes, how they managed it all and through an acceptance of this as a complex problem that can be dealt with, with practice and thought.
I have written previously about classifying plagiarists into three groups (accidental, panicked and systematic), trying to get the student to focus on the journey rather than the objective, and how overwork can produce situations in which human beings do very strange things. Recently, I was asked to sit in on another plagiarism hearing and, because I’ve been away from the role of Assessment Coordinator for a while, I was able to look at the process with an outsider’s eye, a slightly more critical view, to see how it measures up.
Our policy is now called an Academic Honesty Policy and is designed to support one of our graduate attributes: “An awareness of ethical, social and cultural issues within a global context and their importance in the exercise of professional skills and responsibilities”. The principles are pretty straight-forward for the policy:
- Assessment is an aid to learning and involves obligations on the part of students to make it effective.
- Academic honesty is an essential component of teaching, learning and research and is fundamental to the very nature of universities.
- Academic writing is evidence-based, and the ideas and work of others must be acknowledged and not claimed or presented as one’s own, either deliberately or unintentionally.
The policy goes on to describe what student responsibilities are, why they should do the right thing for maximum effect of the assessment and provides some handy links to our Writing Centre and applying for modified arrangements. There’s also a clear statement of what not to do, followed by lists of clarifications of various terms.
Sitting in on a hearing, looking at the process unfolding, I can review the overall thrust of this policy and be aware that it has been clearly identified to students that they must do their own work but, reading through the policy and its implementation guide, I don’t really see what it provides to sufficiently scaffold the process of retraining or re-educating students if they are detected doing the wrong thing.
There are many possible outcomes from the application of this policy, starting with “Oh, we detected something but we turned out to be wrong”, going through “Well, you apparently didn’t realise so we’ll record your name for next time, now submit something new ” (misunderstanding), “You knew what you were doing so we’re going to give you zero for the assignment and (will/won’t) let you resubmit it (with a possible mark cap)” (first offence), “You appear to make a habit of this so we’re giving you zero for the course” (second offence) and “It’s time to go.” (much later on in the process after several confirmed breaches).
Let me return to my discussions on load and the impact on people from those earlier posts. If you accept my contention that the majority of plagiarism cheating is minor omission or last minute ‘helmet fire’ thinking under pressure, then we have to look at what requiring students to resubmit will do. In the case of the ‘misunderstanding’, students may also be referred to relevant workshops or resources to attend in order to improve their practices. However, considering that this may have occurred because the student was under time pressure, we have just added more work and a possible requirement to go and attend extra training. There’s an old saying from Software Development called Brook’s Law:
“…adding manpower to a late software project makes it later.” (Brooks, Mythical Man Month, 1975)
In software it’s generally because there is ramp up time (the time required for people to become productive) and communication overheads (which increases with the square of the number of people again). There is time required for every assignment that we set which effectively stands in for the ramp-up and, as plagiarising/cheating students have probably not done the requisite work before (or could just have completed the assignment), we have just added extra ramp-up into their lives for any re-issued assignments and/or any additional improvement training. We have also greatly increased the communication burden because the communication between lecturers and peers has implicit context based on where we are in the semester. All of the student discussion (on-line or face-to-face) from points A to B will be based around the assignment work in that zone and all lecturing staff will also have that assignment in their heads. An significantly out-of-sequence assignment not only isolates the student from their community, it increases the level of context switching required by the staff, decreasing the amount of effective time that have with the student and increasing the amount of wall-clock time. Once again, we have increased the potential burden on a student that, we suspect, is already acting this way because of over-burdening or poor time management!
Later stages in the policy increase the burden on students by either increasing the requirement to perform at a higher level, due to the reduction of available marks through giving a zero, or by removing an entire course from their progress and, if they wish to complete the degree, requiring them to overload or spend an additional semester (at least) to complete their degree.
My question here is, as always, are any of these outcomes actually going to stop the student from cheating or do they risk increasing the likelihood of either the student cheating or the student dropping out? I complete agree with the principles and focus of our policy, and I also don’t believe that people should get marks for work that they haven’t done, but I don’t see how increasing burden is actually going to lead to the behaviour that we want. (Dan Pink on TED can tell you many interesting things about motivation, extrinsic factors and cognitive tasks, far more effectively than I can.)
This is, to many people, not an issue because this kind of policy is really treated as being punitive rather than remedial. There are some excellent parts in our policy that talk about helping students but, once we get beyond the misunderstanding, this language of support drops away and we head swiftly into the punitive with the possibility of controlled resubmission. The problem, however, is that we have evidence that light punishment is interpreted as a licence to repeat the action, because it doesn’t discourage. This does not surprise me because we have made such a risk/reward strategy framing with our current policy. We have resorted to a punishment modality and, as a result, we have people looking at the punishments to optimise their behaviour rather than changing their behaviour to achieve our actual goals.
This policy is a strange beast as there’s almost no way that I can take an action under the current approach without causing additional work to students at a time when it is their ability to handle pressure that is likely to have led them here. Even if it’s working, and it appears that it does, it does so by enforcing compliance rather than actually leading people to change the way that they think about their work.
My conjecture is that we cannot isolate the problems to just this policy. This spills over into our academic assessment policies, our staff training and our student support, and the key difference between teaching ethics and training students in ethical behaviour. There may not be a solution in this space that meets all of our requirements but if we are going to operate punitively then let us be honest about it and not over-burden the student with remedial work that they may not be supported for. If we are aiming for remediation then let us scaffold it properly. I think that our policy, as it stands, can actually support this but I’m not sure that I’ve seen the broad spread of policy and practice that is required to achieve this desirable, but incredibly challenging, goal of actually changing student behaviour because the students realise that it is detrimental to their learning.
The session kicked off with “The Abstraction Transition Taxonomy: Developing Desired Learning Outcomes through the Lens of Situated Cognition”, (Quintin Cutts (presenting), Sarah Esper, Marlena Fecho, Stephen Foster and Beth Simon) and the initial question: “Do our learning outcomes for programming classes match what we actually do as computational thinkers and programmers?” To answer this question, we looked Eric Mazur’s Peer Instruction, an analysis of PU questions as applied to a CS principles pilot course, and then applied the Abstraction Transition Taxonomy (ATT) to published exams, with a wrap of observations and ‘where to from here’.
Physicists have, some time ago, noticed that their students can plug numbers into equations (turn the handle, so to speak) but couldn’t necessarily demonstrate that they understood things: they couldn’t demonstrate that that they thought as physicists should. (The Force Concept Inventory was mentioned here and, if you’re not familiar, it’s a very interesting thing to look up.) To try and get students who thought as physicists, Mazur developed Peer Instruction (PI), which had pre-class prep work, in-class questions, followed by voting, discussion and re-voting, with an instructor leading class-wide discussion. These activities prime the students to engage with the correct explanations – that is, the way that physicists think about and explain problems.
Looking at Computer Science, many CS people use the delivery of a working program as a measure of the correct understanding and appropriate use of programming techniques.
Given that generating a program is no guarantee of understanding, which is sad but true given the existence of the internet, other students and books. We could try and force a situation where students are isolated from these support factors but this then leads us back to permutation programming, voodoo code and shotgun debugging unless the students actually understand the task and how to solve it using our tools. In other words, unless they think as Computer Scientists.
UCSD had a CS Principles Pilot course that used programming to foster computational thinking that was aimed at acculturation into the CS ‘way’ rather than trying to create programmers. The full PI implementation asked students to reason about their programs, through exploratory homework and a PI classroom, with some limited time traditional labs as well. While this showed a very positive response, the fear was that this may have been an effect of the lecturers themselves so analysis was required!
By analysing the PI questions, a taxonomy was developed that identified abstraction levels and the programming concepts within them. The abstraction levels were “English”, “Computer Science Speak” and “Code”. The taxonomy was extended with the transitions between these levels (turning an English question into code for example is a 1-3 transition, if English is abstraction level 1 and Code 3. Similarly, explain this code in English is 3-1). Finally, they considered mechanism (how does something work) and rationale (why did we do it this way)?
Analysing the assignment and assessment questions to determine what was being asked, in terms of abstraction level and transitions, and whether it was mechanism or rationale, revealed that 21% of the in-class multiple choice questions were ‘Why?’ questions but there actually very few ‘Why?’ questions in the exam. Unsurprisingly, almost every question asked in the PI framework is a ‘Why?’ question, so there should be room for improvement in the corresponding examinations. PI emphasises the culture of the discipline through the ‘Why?’ framing because it requires acculturation and contextualisation to get yourself into the mental space where a Rationale becomes logical.
The next paper “Subgoal-Labeled Instructional Material Improves Performance and Transfer in Learning to Develop Mobile Applications”, Lauren Margulieux, Mark Guzdial and Richard Catrambone, dealt with mental models and how the cognitive representation of an action will affect both the problem state and how well we make predictions. Students have so much to think about – how do they choose?
The problem with just waiting for a student to figure it out is high cognitive load, which I’ve referred to before as helmet fire. If students become overwhelmed they learn nothing, so we can explicitly tell students and/or provide worked examples. If we clearly label the subgoals in a worked example, students remember the subgoals and the transition from one to another. The example given here was an Android App Inventor worked example, one example of which had no labels, the other of which had subgoal labels added as overlay callouts to the movie as the only alteration. The subgoal points were identified by task analysis – so this was a very precise attempt to get students to identify the important steps required to understand and complete the task.
(As an aside, I found this discussion very useful. It’s a bit like telling a student that they need comments and so every line has things like “x=3; //x is set to 3” whereas this structured and deliberate approach to subgoal definition shows students the key steps.)
In the first experiment that was run, the students with the subgoals (and recall that this was the ONLY difference in the material) had attempted more, achieved more and done it in less time. A week later, they still got things right more often. In the second experiment, a talk-aloud experiment, the students with the subgoals discussed the subgoals more, tried random solution strategies less and wasted less effort than the other group. This is an interesting point. App Inventor allows you to manipulate blocks of code and the subgoal group were less likely to drag out a useless block to solve the problem. The question, of course, is why. Was it the video? Was it the written aspects? Was it both?
Students appear to be remembering and using the subgoals and, as was presented, if performance is improving, perhaps the exact detail of why it’s happening is something that we wish to pursue but, in the short term, we can still use the approach. However, we do have to be careful with how many labels we use as overloading visual cues can lead to confusion, thwarting any benefit.
The final paper in the session was “Using collaboration to overcome disparities in Java experience”, Colleen Lewis (presenting), Nathaniel Titterton and Michael Clancy. This presented the transformation of a a standard 3 Lecture, 2 hours of lab and 1 discussion hour course into a 1 x 1 hour lecture with 2 x 3 hour labs, with the labs now holding the core of the pedagogy. Students are provided feedback through targeted tutoring, using on-line multiple choices for the students to give feedback and assist the TAs. Pair programming gives you someone to talk to before you talk to the TA but the TA can monitor the MCQ space and see if everyone is having a problem with a particular problem.
This was addressing a problem in a dual speed entry course, where some students had AP CS and some didn’t, therefore the second year course was either a review for those students who had Java (from AP CS) or was brand new. Collaboration and targeted support was aimed at reducing the differences between the cohorts and eliminate disadvantage.
Now, the paper has a lot of detail on the different cohorts, by intake, by gender, by retention pattern, but the upshot is that the introduction of the new program reduced the differences between those students who did and did not have previous Java experience. In other words, whether you started at UCB in CS 1 (with no AP CS) or CS 1.5 (with AP CS), the gap between your cohorts shrank – which is an excellent result. Once this high level of collaboration was introduced, the only factor that retained any significant difference was the first exam, but this effect disappeared throughout the course as students received more exposure to collaboration.
I strongly recommend reading all three of these papers!
One of the things about being a Computer Science researcher who is on the way to becoming a Computer Science Education Researcher is the sheer volume of educational literature that you have to read up on. There’s nothing more embarrassing than having an “A-ha!” moment that turns out to have been covered 50 years and the equivalent of saying “Water – when it freezes – becomes this new solid form I call Falkneranium!”
Ahem. So my apologies to all who read my ravings and think “You know, X said that … and a little better, if truth be told.” However, a great way to pick up on other things is to read other people’s blogs because they reinforce and develop your knowledge, as well as giving you links to interesting papers. Even when you’ve seen a concept before, unsurprisingly, watching experts work with that concept can be highly informative.
I was reading Mark Guzdial’s blog some time ago and his post on the Khan Academy’s take on Computer Science appealed to me for a number of reasons, not least for his discussion of scaffolding; in this case, a tutor-guided exploration of a space with students that is based upon modelling, coaching and exploration. Importantly, however, this scaffolding fades over time as the student develops their own expertise and needs our help less. It’s like learning to ride a bike – start with trainer wheels, progress to a running-alongside parent, aspire to free wheeling! (But call a parent if you fall over or it’s too wet to ride home.)
One of my key areas of interest is self-regulation in students – producing students who no longer need me because they are self-aware, reflective, critical thinkers, conscious of how they fit into the discipline and (sufficiently) expert to be able to go out into the world. My thinking around Time Banking is one of the ways that students can become self-regulating – they manage their own time in a mature and aware fashion without me having to waggle a finger at them to get them to do something.
Today, R (postdoc in the Computer Science Education Research Group) and I were brainstorming ideas for upcoming papers over about a 2 hour period. I love a good brainstorm because, for some time afterwards, ideas and phrases come to me that allow me to really think about what I’m doing. Combining my reading of Mark’s blog and the associated links, especially about the deliberate reduction of scaffolding over time, with my thoughts on time management and pedagogy, I had this thought:
If imposed deadlines have any impact upon the development of student timeliness, why do we continue to need them into the final year of undergraduate and beyond? When do the trainer wheels come off?
Now, of course, the first response is that they are an administrative requirement, a necessary evil, so they are (somehow) exempt from a pedagogical critique. Hmm. For detailed reasons that will go into the paper I’m writing, I don’t really buy that. Yes, every course (and program) has a final administrative requirement. Yes, we need time to mark and return assignments (or to provide feedback on those assignments, depending on the nature of the assessment obviously). But all of the data I have says that not only do the majority of students hand up on the last day (if not later), but that they continue to do so into later years – getting later and later as they progress, rather than earlier and earlier. Our administrative requirement appears to have no pedagogical analogue.
So here is another reason to look at these deadlines, or at least at the way that we impose them in my institution. If an entry test didn’t correlate at all with performance, we’d change it. If a degree turned out students who couldn’t function in the world, industry consultation would pretty smartly suggest that we change it. Yet deadlines, which we accept with little comment most of the time, only appear to work when they are imposed but, over time, appear to show no development of the related skill that they supposedly practice – timeliness. Instead, we appear to enforce compliance and, as we would expect from behavioural training on external factors, we must continue to apply the external stimulus in order to elicit the appropriate compliance.
Scaffolding works. Is it possible to apply a deadline system that also fades out over time as our students become more expert in their own time management?
I have two days of paper writing on Thursday and Friday and ‘m very much looking forward to the further exploration of these ideas, especially as I continue to delve into the deep literature pile that I’ve accumulated!
I’m a bit of a hypocrite when it comes to taking my own advice on exercise. “Start easy,” I say, as I come back from injury with a fast 10 mile run. “Don’t over do it!” I admonish as I try to lift my own body weight in a gym. I have two knee surgeries, multiple calf strains and a torn plantar fascia to bear witness to this. For this, and many other reasons, I have a personal training for the gym stuff who I see once a week and a running partner who (facetiously) threatens to slap me if I run too hard or too far. I’m more scared of her than I am my personal trainer but don’t tell her that.
To fit everything in, I train in the gym at 6am on Wednesdays and, while this gives me a long day it allows me to get some good core and upper body work in to balance my legs and overall fitness – strength is useful in many ways as a day-to-day thing but balance, the other effect of good core strength, is incredibly useful and makes me a much better runner. I also take the opportunity to talk to someone who doesn’t work with me, isn’t related to me and knows what I do, but not in much detail. It’s very relaxing to talk to someone like that, especially when you have things on your mind.
Time Banking has been on my mind, as has my own time management, so I’ve talked a lot about making my life more effective, working less and working better, all those good things. So imagine my surprise when my trainer wrote to me asking if we could move sessions a little, if possible, as he’d been listening to what I’d been saying and realised that he’d been working reactively by not allowing himself enough time to plan and structure his day, especially as he’s now managing the gym I train at.
I’ve been thinking about changing away from the Wednesday 6 slot and this means that this is great timing for me. I’d like to keep training with him but I could train with someone else who doesn’t have his burdens or schedule quite easily and still have a really good experience – I’ve trained with other people before there, it’s a good gym. But what I really like is the thought process – contemplative and transformative.
Now, either this is the greatest con job that I’ve been privy to or I may have actually helped someone else to see a new way of thinking about their own life. Either way, there appears to be some knowledge transfer going on.
One person at a time. It’s slow but I can work with that. 🙂
Short post today. I’m reminded that, even with the best view of what you have to do, any time that you have in your calendar to do things can easily disappear when the unexpected strikes. I had this week planned as a reasonably paced week, with some paper writing. Now I’m looking at a week where I have one unscheduled 30 minute period until Friday evening.
Was this poor planning? No, I had my calendar planned with preparation time and all days were sitting under at my 70% scheduled limit, well under, in fact, because I wanted to allow as much drop-in time as possible for my students. However, now the time has filled up and, yet, all of my deadlines for this week still apply – plus some more on top.
It’s a reminder that stuff happens sometimes and, as we (eventually) start writing the time banking papers, it’s important to remember how easy it is to go from “everything’s cool” to “oh no, my brain is on fire!”. Now I have a lot of experience in handling brain fire but, even so, it’s that nasty little shock that means that there will be no early nights for me until Sunday. This is a surge week, a stretch week, so the usual 45 hour upper limit (the guideline to see if I can get everything done) is on hiatus but will be reset for Monday.
Now this is important because I have, so far, been able to get everything done within the time that I’ve been allocating, I’ve been more relaxed and I’ve been more effective. Today wasn’t the best day in some ways because the wheels had started to fall off and most of my day was spent planning in one part of my head while working with the other.
I see students hit this point a lot and an important part of my job is talking them down from the ledge, in effect. It’s very hard for people who are working o hard, and not getting everything done, to think that working fewer hours will allow them to achieve their goals. So far, I’ve been able to do it, as long as I am open to the occasional burst of week but, at the moment, I’m sitting the limit at 1 week.
We’ll see if that works!