The Complex Roles Of Universities In The Period Of Globalization – Altbach – Part 1
Posted: September 23, 2012 Filed under: Education | Tags: advocacy, authenticity, community, education, educational problem, educational research, ethics, Generation Why, grand challenge, higher education, in the student's head, learning, measurement, resources, student perspective, teaching, teaching approaches, thinking, tools Leave a commentOne of the most handy things about having a new member in a research group, especially one who is just finishing or has just finished submitting a PhD, is that they come in with an entirely new subset of the possible papers in the given discipline, which they have used to construct their theses and inform their thinking. While you will have the standard overlap of the key papers in the field, there will often be waterways that run away from the main river and it is in these diverse streams that we find new ways of thinking, even leading to these stream becoming tributaries that feed back into our main body, strengthening the overall work.
R has just sent me a reference from her thesis, a copy of Altbach’s 2011 publication in Higher Education in the World 3: New Challenges and Emerging Roles for Human and Social Development, entitled “The Complex Roles Of Universities In The Period Of Globalization”. The abstract is pithy but this quote stands out: “The academic drift of the 21st century raises concerns about the core functions of universities and how contemporary changes have affected academic missions.” This is a fascinating paper and one that I wish I turned up early because it has the same concerns as I do, and as Richard Hil did with his Whackademia book, in that we are all being asked to do more with less and it is how we do this that will decide our future, and the future of higher education. (Readers may recall that I did not agree with much of what Hil said – as I said, I wish I had read Altbach sooner because it would have made the rebuttal easier.) I’m going to cover this across a few posts because the paper has a fair bit of comment and I’d like to make some commentary!
Altbach looks at the different roles that Universities have had over time, including the different roles that they play in certain countries and how time, politics, religion, wealth and nationalism have all contributed to changing demands on the sector. There is no doubt that teaching and research make up our core functions but we can vary from country to country as to whether we are teaching technical skills, professional skills or general education at Universities. Over time, we have often been in conflict with our own societies, which can lead to great creativity but at the cost of additional load or difficult burdens. Research is equally difficult to pin down: are we talking ‘pure’ research or ‘applied’ research? Does research have to be discipline focused or can we perform research on teaching, or research on research? Does it matter where the research money comes from? Different areas inside the same university can have completely different answers to these questions so it’s little doubt that this question is still open!
Universities have been used to foster national development and identity, as Altbach mentions with German, Japanese and American examples, or as stabilising influences in the third world. We are also steadily evolving academic centres, adding courses as the ranks of the professions grow. My profession, Computer Scientist, wasn’t even a profession until the second half of the 20th century (that’s why we have so few cool awards – there is no Nobel prize in Applied Algorithmics). Immediately we see a conflict in the sense of stability and status quo required to be a national touchstone, while determining how we adapt to the changing demands of the workforce and the new professionals.
We have always been associated with knowledge as both the defenders and disseminators, ignoring secular and religious demand to not teach certain things or to state that red is black, with a focus on organisation to facilitate later retrieval. This access to knowledge also feeds in to one of our other key facets, or at least one of the most desirable, that of an intellectual centre. As academics, we have the freedom to express our ideas and, many would argue, the obligation to do so given that we have that freedom. The expertise that our staff have should be available to all in terms of interpretation and refinement of ideas and concepts but to do that we have to engage with the community. It is of little surprise that we often find ourselves involved in social and political movements, supporting other activists, providing resources and making an overall contribution to the intellectual life of our surroundings.
This is, for me, a very important point because it forces us to consider where the private individual ends and the public intellectual begins, if such a division even makes any sense. From a personal perspective, I would not raise my politics in a classroom but I would discuss issues of ethics and equality, some of which may or may not be in accord with prevailing government thought. Let me be more explicit. Yesterday, I attended a rally for Marriage Equality, as part of a reaction against the Australian Federal Government’s rejection of a bill to allow same sex marriage. I would most certainly not have advertised this event in my lectures or told my students about it because I think that there’s far too much capacity for me to influence my students to act through our relationship, which is not a discussion or political sharing but overt influence. I attended the rally as a private citizen but if my students asked me about it, because we did get photographed and videoed, then I feel that I could explain my actions within an ethical framework, which means that this is informing my role as public intellectual. My community, equality and ethical focus drives both the citizen and the academic and allows me to carry out two roles while attempting to minimise any exploitation of the power relationship that I have with my students. However, my capacity as a (notional) public intellectual requires me to have an explanation for what I did that is articulate and comprehensible. The private citizen is impassioned but the academic is both passionate and rationale, and can place the activity in a context that allows it to be shared.
But, as I always say, there is no point having a system that only works with perfect people. Altbach is talking about our institutions, which is the right focus for the paper, but the institutions are just buildings without the academics and students that fill them. I attempt to juggle my private and public self and, while sometimes I succeed more than others, I think I know what I should ‘look like’ to my institution, my peers, my students and my social groups. What will be interesting in the coming world of change for Universities is how we deal with the people who don’t work as well within the role of educator. I have no time, respect or tolerance for those of my colleagues who confuse intellectual freedom with a wanton disregard for reasonable behaviour in this privileged role. Just because we organise the knowledge doesn’t mean that we own it, nor does our mastery of intellectual pursuits (if we achieve that) make us any better than anyone else: we have merely had more opportunity but, for me, that comes with a corresponding level of responsibility. I have seen more than one academic, not at my own University I hasten to add, who has obviously been grooming a student through manipulation of the aura of competency that any decent academic can muster, where we appear wise, worldly and incredibly, staggeringly, deep on matters that are so very, very passionate and important. Altbach writes of what changes we have seen in Universities but you only have to read through the yellow press (or the FFFF00 press on the web) to see how many educators are abusing their relationship with their students and I’m not sure what this says about how the educators themselves are changing. I have heard dire tales of exploitative behaviour in the 70s and 80s in my country – “A for a lay” unpleasantness and similar. When we talk of our intellectual freedoms, our influence on the world as national stabilisers and centres of knowledge, it is important to remember that the components of these institutions are merely people. As we increase the stresses on the organisations, so too do we distribute this across people and, given that people are already failing some key moral and disciplinary requirements, any discussion of what our role should become will have to take into account the fact that we are building a system from people, to work with other people.
The Philosophical Angle
Posted: September 21, 2012 Filed under: Education | Tags: advocacy, community, curriculum, education, educational problem, educational research, ethics, feedback, Generation Why, higher education, identity, in the student's head, learning, philosophy, reflection, resources, seneca, socrates, stoicism, student perspective, teaching, teaching approaches, thinking, universal principles of design Leave a commentSocrates drank hemlock after being found guilty of corrupting the minds of the youth of Athens, and impiety. Seneca submitted to the whims of Nero when the Emperor, inevitably, required that his old tutor die. Seneca’s stoicism was truly tested in this, given that he slashed his veins, took poison, jumped in a warm bath and finally had to be steamed to death before Nero’s edict that he kill himself was finally enacted. I, fortunately, expect no such demonstrations of stoic fortitude from my students but, if we are to think about their behaviour and development as self-regulating beings, then I think that a discussion of their personal philosophy becomes unavoidable. We have talked about the development state, their response to authority, their thoughts on their own thinking, but what of their philosophy?
If you are in a hurry and jump in your car, every red light between you and your destination risks becoming a personal affront, an enraging event that defies your expectation of an ‘all-green’ ride into town. There is no reason why you should expect such favours from the Universe, whatever your belief system, but the fact that this is infuriating to you remains. In the case of the unexpected traffic light, which sounds like the worst Sherlock Holmes story ever, the worst outcome is that you will be late, which may have a variety of repercussions. In preparing assignment work, however, a student may end up failing with far more dire and predictable results.

“Watson, I shall now relate the entire affair through Morse tapped pipe code and interpretative dance.”
While stoicism attracts criticism, understandably, because it doesn’t always consider the fundamentally human nature of humans, being prepared for the unforeseen is a vital part of any planning process. Self-regulation is not about drawing up a time table that allows you to fit in everything that you know about, it is about being able to handle your life and your work when things go wrong. Much as a car doesn’t need to be steered when it is going in a straight line and meeting our requirements, it is how we change direction when we know the road and when a kangaroo jumps out that are the true tests of our ability to manage our resources and ourselves.
Planning is not everything, as anyone who has read Helmuth von Moltke the Elder or von Clausewitz will know: “no plan survives contact with the enemy”. In this case, however, the enemy is not just those events that seek to confound us, it can be us as well! You can have the best plan in the world that relies upon you starting on Day X, and yet you don’t. You may have excellent reasons for this but, the fact remains, you have now introduced problems into your own process. You have met the enemy and it is you. This illustrates the critical importance of ensuring that we have an accurate assessment of our own philosophies – and we do have to be very honest.
There is no point in a student building an elaborate time management plan that relies upon them changing the habits of a lifetime in a week. But this puts the onus upon us as well: there is no point in us fabricating a set of expectations that a student cannot meet because they do not yet have a mature philosophy for understanding what is required. We don’t give up (of course!) but we must now think about how we can scaffold and encourage such change in a manageable way. I find reflection very handy, as I’ve said before, as watching students write things like “I planned for this but then I didn’t do it! WHY?” allows me to step in and discuss this at the point that the student realises that they have a problem.
I am not saying that a student who has a philosophy of “Maybe one day I will pass by accident” should be encouraged to maintain such lassitude, but we must be honest and realise that demanding that their timeliness and process maturity spring fully-formed from their foreheads is an act of conjuring reserved only for certain Greek Gods. (Even Caligula couldn’t manage it and he had far greater claim to this than most.) I like to think of this in terms of similarity of action. If anything I do is akin to walking up to someone and yelling “You should hand in on time, do better!” then I had better re-think my strategy.
The development of a personal philosophy, especially when you may not have ever been exposed to some of the great exemplars, is a fundamentally difficult task. You first need to understand that such a concept exists, then gain the vocabulary for discussing it, then interpret your current approach and see the value of change. Once you have performed all of those tasks, then we can start talking about getting from A to B. If you don’t know what I’m talking about or can’t understand why it’s important, or even discuss core concepts, then I’m yelling at you in the corridor and you’ll nod, compliantly, until I go away. Chances of you taking positive steps in the direction that I want? Very low. Probably, nil. And if it does happen, either it’s accidental or you didn’t actually need my help.
I try to be stoic but I must be honest and say that if Nero sentenced me to death, I’d nod, say “I expected that”, then put on some fast saxophone music and leg it up over the seven hills and far away. I don’t think I’d ever actually expect true stoicism from most of my students. but a simple incorporation of the fact that not everything works out as you think it will would be a definite improvement over the current everything will work out in my favour expectation that seems to be the hallmark of the more frequently disappointed and distressed among them. The trick is that I first have to make them realise that this is something that, with thought, they can not only fix but use to make a genuine, long-lasting and overwhelmingly positive change in their lives.
ICER 2012 Day 3 Research Paper Session 5
Posted: September 15, 2012 Filed under: Education | Tags: advocacy, authenticity, community, education, educational research, ethics, Generation Why, higher education, icer, icer 2012, icer2012, in the student's head, learning, negative perceptions, teaching, teaching approaches, tools, universal principles of design Leave a commentThe last of the research paper sessions and, dear reader, I am sure that you are as glad as I that we are here. Reading about an interesting conference that you didn’t attend is a bit like receiving a message from a friend talking about how he kissed the person that you always loved from afar. Thanks for the information but I would rather have been there myself.
This session opened with “Toward a Validated Computing Attitudes Survey” (Allison Elliott Tew, Brian Dorn and Oliver Schneider), where the problems with negative perceptions of the field and hostile classroom environments, combined with people thinking that they would be no good at CS, conspire to prevent students coming in to, or selecting, our discipline. The Computing Attitudes Survey was built, with major modification, from the Colorado Learning Attitudes about Science Survey (CLASS, pronounced C-LASS). To adapt the original survey, some material was just copied across with a word change (computer science replacing physics), some terminology was changed (algorithm for formula) and some discipline specific statements were added. Having established an expert opinion basis for the discipline specific content, students can now see how much they agree with the experts.
There is, as always, the rip of contentious issues. “You have to know maths to be able to program” was a three-way split within the expert group as to who agreed, disagreed or was neutral. What was interesting, and what I’ll be looking at in future, is the evidence of self-defeating thought in many answers (no, not questions. The questions weren’t self-defeatist but the answers often were.) What was also interesting is that attitudes seem to get worse in the CLASS instrument after you take the course!
Confidence, as simple as “I think I can do this”, plays a fundamental part in determining how students will act. Given the incredibly difficult decisions that a student faces when selecting their degree or concentration, it is no surprise that anyone who thinks “Computing is too hard for me” or “Computing is no use to me” will choose to do something else.
The authors are looking for volunteers where they can run these trials again so, after you’ve read their paper, if you’re interested, you should probably e-mail them.
“A Statewide Survey on Computing Education Pathways and Influences: Factors in Broadening Participation in Computing” (Mark Guzdial, Barbara Ericson, Tom McKlin and Shelly Engelman)
The final research paper in the conference dealt with the final evaluation of the Georgia Computes! initiative, which had run from October 2006 to August of this year. This multi-year project cannot be contained in my nervous babbling but I can talk about the instrument that was presented. Having run summer camps, weekend workshops, competitions, teacher workshops, a teachers’ lending library, first year engagement and seeded first-year summer camps (whew!), the question was: What had been the impact of Georgia Computes! ? What factors influence undergrad enrolment into intro CS courses?
There were many questions and results presented but I’d like to focus on the top four reasons given, from survey, as to why students weren’t going to undertake a CS Major or Minor:
- I don’t want to do the type of work
- Little interest in the subject matter
- Don’t enjoy Computing Courses
- Don’t have confidence that I would succeed.
Looking at those points, after a state-wide and highly successful campaign over 6 years has finished, it is very, very sobering for me. What these students are saying is that they cannot see the field as attractive, interesting, enjoyable or that they are capable. But these are all aspects that we can work on, although some of these will require a lot of work.
Two further things that Barb said really struck me. Firstly, that if you take into account encouragement and ability, that men will tend to be satisfied and continue on if they receive either or both – the factors are not separable for men – but that women and minorities need encouragement in order to feel satisfied and to convince them to keep going. Secondly, when it comes to giving encouragement, male professors are just as effective as female professors in terms of giving encouragement to women.
As a male lecturer, who is very, very clearly aware of the demographic disgrace that is the under-representation of women in CS, this first fact gives me a partial strategy to increase retention (and reinforces a believe I have held anecdotally for some time) but the second fact gives me the agency to assist in this process, as well as greater hope for a steadily increasing female cohort over time.
Overall, a very positive note on which to finish the session papers!
The Narrative Hunger: Stories That Meet a Need
Posted: September 15, 2012 Filed under: Education | Tags: authenticity, collaboration, community, curriculum, design, education, educational research, feedback, Generation Why, higher education, in the student's head, learning, principles of design, resources, student perspective, teaching, teaching approaches, tools, universal principles of design Leave a commentI have been involved in on-line communities for over 20 years now and, apparently, people are rarely surprised when they meet me. “Oh, you talk just like you type.” is the effective statement and I’m quite happy with this. While some people adopt completely different personae on-line, for a range of reasons, I seem to be the same. It then comes as little surprise that I am as much of storyteller in person as I am online. I love facts, revel in truth, but I greatly enjoying putting them together into a narrative that conveys the information in a way that is neither dry nor dull. (This is not to say that the absence of a story guarantees that things must be dry and dull but, without a focus on those elements of narrative that appeal to common human experience, we always risk this outcome.)
One of Katrina’s recent posts referred to the use of story telling in education. As she says, this can be contentious because:
stories can be used to entertain students, to have them enjoy your lectures, but are not necessarily educational.
The shibboleth of questionable educational research is often a vaguely assembled study, supported by the conjecture that the “students loved it”, and it is very easy to see how story telling could fall into this. However, we as humans are fascinated by stories. We understand the common forms even where we have not read Greek drama or “The Hero With a Thousand Faces”. We know when stories ring true and when they fall flat. Searching the mental engines of our species for the sweet spots that resonate across all of us is one way to convey knowledge in a more effective and memorable way. Starting from this focus, we must then observe our due diligence in making sure that our story framework contains a worthy payload.
I love story telling and I try to weave together a narrative in most of my lectures, even down to leaving in sections where deliberate and tangential diversion becomes part of the teaching, to allow me to contrast a point or illuminate it further by stripping it of its formal context and placing it elsewhere. After all, an elephant next to elephants is hardly memorable but an elephant in a green suit, as King of a country, tends to stick in the mind.
The power of the narrative is that it involves the reader or listener in the story. A well-constructed narrative leads the reader to wonder about what is going to happen next and this is model formation. Very few of us read in a way where the story unfolds with us completely distant from it – in fact, maintaining distance from a story is a sign of a poor narrative. When the right story is told, or the right person is telling it, you are on the edge of your seat, hungry to know more. When it is told poorly, then you stifle a yawn and smile politely, discreetly peering at your watch as you attempt to work out the time at which you can escape.
Of course, this highlights the value of narrative for us in teaching but it also reinforces that requirement that it be more than an assemblage of rambling anecdotes, it must be a constructed narration that weaves through points in a recognisable way and giving us the ability to conjecture on its direction. O. Henry endings, the classic twist endings, make no sense unless you have constructed a mental model that can be shaken by the revelations of the last paragraphs. Harry Potter book 7 makes even less sense unless one has a model of the world in which the events of the book can be situated.
As always, this stresses the importance of educational design, where each story, each fact, each activity, is woven into the greater whole with a defined purpose and in full knowledge of how it will be used. There is nothing more distracting than someone who rambles during a lecture about things that not only seem irrelevant, but are irrelevant. Whereas a musing on something that, on first glance, appears irrelevant can lead to exploration of the narrative by students. Suddenly, they are within a Choose Your Own Adventure book and trying to work out where each step will take them.
Stories are an excellent way to link knowledge and problems. They excite, engage and educate, when used correctly. We are all hungry for stories: we are players within our own stories, observers of those of the people around us and, eventually, will form part of the greater narrative by the deeds for which we are written up in the records to come. It makes sense to use this deep and very human aspect of our intellect to try and assist with the transfer of knowledge.
Our Influence: Prejudice As Predictor
Posted: September 14, 2012 Filed under: Education | Tags: advocacy, authenticity, community, education, educational problem, educational research, higher education, in the student's head, learning, measurement, principles of design, reflection, student perspective, teaching, teaching approaches, thinking Leave a commentIf you want to see Raymond Lister get upset, tell him that students fall into two categories: those who can program and those who can’t. If you’ve been reading much (anything) of what I’ve been writing recently, you’ll realise that I’ve been talking about things like cognitive development, self-regulation, dependence on authority, all of which have one thing in common in that students can be at different stages when they reach us. There is no guarantee that students will be self-reliant, cognitively mature and completely capable of making reasoned decisions at the most independent level.
There was a question raised several times during the conference and it’s the antithesis of the infamous “double hump conjecture”, that students divide into two groups naturally and irrevocably because of some innate characteristic. The question is “Do our students demonstrate their proficiency because of what we do or in spite of what we do?” If the innate characteristic conjecture is correct, and this is a frequently raised folk pedagogy, then our role has no real bearing on whether a student will learn to program or not.
If we accept that students come to us at different stages in their development, and that these development stages will completely influence their ability to learn and form mental models, then the innate characteristic hypothesis withers and dies almost immediately. A student who does not have their abilities ready to display can no more demonstrate their ability to program than a three-year old child can write Shakespeare – they are not yet ready to be able to learn, assemble, reassemble or demonstrate the requisite concepts and related skills.
However, a prejudicial perspective that students who cannot demonstrate the requisite ability are innately and permanently lacking that skill will, unpleasantly, viciously and unnecessarily, cause that particular future to lock in. Of course a derisive attitude to these ‘stupid’ or ‘slow’ students will make them withdraw or undermine their confidence! As I will note from the conference, confidence and support have a crucial impact on students. Undermining a student’s confidence is worse than not teaching them at all. Walking in with the mental model that separates the world into programmers and non-programmers forces that model into being.
Since I’ve entered the area of educational research, I’ve been exposed to things that I can separate into the following categories:
- Fascinating knowledge and new views of the world, based on solid research and valid experience.
- Nonsense
- Damned nonsense
- Rank stupidity
Where most of the latter come from other educators who react, our of fear or ignorance, to the lessons from educational research with disbelief, derision and resentment. “I don’t care what you say, or what that paper says, you’re wrong” says the voice of “experience”.
There is no doubt that genuine and thoughtful experience is, has been, and will always be a strong and necessary sibling to the educational and psychological theory that is the foundation of educational research. However, shallow experience can often be built up into something that it is not, when it is combined with fallacious thinking, cherry picking, confirmation bias and any other permutation of fear, resentment and inertia. The influence of folk pedagogies, lessons claimed from tea room mutterings and the projection of a comfortable non-reality that mysteriously never requires the proponent to ever expend any additional effort or change what they do, is a malign shadow over the illumination of good learning and teaching practice.
The best educators explain their successes with solid theory, strive to find a solution to the problems that lead to failure, and listen to all sources in order to construct a better practice and experience for their students. I hope, one day, to achieve this level- but I do know that doubting everything new is not the path forward for me.
I am pleased to say that the knowledge and joy of this (to me) new field far outstrips most of the other things that I have seen but I cannot stress any more how important it is that we choose our perspectives carefully. We, as educators, have disproportionally high influence: large shadows and big feet. Reading further into this discipline illustrates that we must very carefully consider the way that we think, the way that our students think and the capability that we actually have in the students for reasoning and knowledge accumulation before we make any rash or prejudicial statements about the innate capabilities of that most mythical of entities: the standard student.
ICER 2012: Day 0 (Workshops)
Posted: September 10, 2012 Filed under: Education | Tags: collaboration, community, design, education, educational problem, educational research, feedback, Generation Why, higher education, icer, icer 2012, in the student's head, learning, principles of design, student perspective, teaching, teaching approaches, workload 1 CommentWell, it’s Sunday so it must be New Zealand (or at least it was Sunday yesterday). I attended that rarest of workshops, one where every session was interesting and made me think – a very good sign for the conference to come.
We started with an on-line workshop on Bloom’s taxonomy, classifying exam questions, with Raymond Lister from UTS. One of the best things about this for me was the discussion about the questions where we disagreed: is this application or synthesis? It really made me think about how I write my examinations and how they could be read.
We then segued into a fascinating discussion of neo-Piagetian theory, where we see the development stages that we usually associate with children in adults as they learn new areas of knowledge. In (very rough) detail, we look at whether we have enough working memory to carry out a task and, if not, weird things happen.
Students can indulge in some weird behaviours when they don’t understand what’s going on. For example, permutation programming, where they just type semi-randomly until their program compiles or works. Other examples include shotgun debugging and voodoo programming and what these amount to are the student not having a good consistent model of what works and, as a result, they are basically dabbling in a semi-magic approach.
My notes from the session contain this following excerpt:
“Bizarro” novice programmer behaviours are actually normal stages of intellectual development.Accept this and then work with this to find ways of moving students from pre-op, to concrete op, to formal operational. Don’t forget the evaluation. Must scaffold this process!
What this translates to is that the strange things we see are just indications that students having moved to what we would normally associate with an ‘adult’ (formal operational) understanding of the area. This shoots several holes in the old “You’re born a programmer” fallacy. Those students who are more able early may just have moved through the stages more quickly.
There was also an amount of derisive description of folk pedagogy, those theories that arise during pontification in the tea room, with no basis in educational theory or formed from a truly empirical study. Yet these folk pedagogies are very hard to shake and are one of the most frustrating things to deal with if you are in educational research. One “I don’t think so” can apparently ignore the 70 years since Dewey called the classrooms prisons.
The worst thought is that, if we’re not trying to help the students to transition, then maybe the transition to concrete operation is happening despite us instead of because of us, which is a sobering thought.
I thought that Ray Lister finished the session with really good thought regarding why students struggle sometimes:
The problem is not a student’s swimming skill, it’s the strength of the torrent.
As I’ve said before, making hard things easier to understand is part of the job of the educator. Anyone will fail, regardless of their ability, if we make it hard enough for them.
Conference Blogging! (Redux)
Posted: September 8, 2012 Filed under: Education | Tags: blogging, education, educational problem, educational research, feedback, Generation Why, higher education, icer, icer 2012, in the student's head, learning, measurement, student perspective, teaching, teaching approaches, time banking, workload 1 CommentI’m about to head off to another conference and I’ve taken a new approach to my blogging. Rather than my traditional “Pre-load the queue with posts” activity, which tends to feel a little stilted even when I blog other things around it, I’ll be blogging in direct response to the conference and not using my standard posting time.
I’m off to ICER, which is only my second educational research conference, and I’m very excited. It’s a small but highly regarded conference and I’m getting ready for a lot of very smart people to turn their considerably weighty gaze upon the work that I’m presenting. My paper concerns the early detection of at-risk students, based on our analysis of over 200,000 student submissions. In a nutshell, our investigations indicate that paying attention to a student’s initial behaviour gives you some idea of future performance, as you’d expect, but it is the negative (late) behaviour that is the most telling. While there are no astounding revelations in this work, if you’ve read across the area, putting it all together with a large data corpus allows us to approach some myths and gently deflate them.
Our metric is timeliness, or how reliably a student submitted their work on time. Given that late penalties apply (without exception, usually) across the assignments in our school, late submission amounts to an expensive and self-defeating behaviour. We tracked over 1,900 students across all years of the undergraduate program and looked at all of their electronic submissions (all programming code is submitted this way, as are most other assignments.) A lot of the results were not that unexpected – students display hyperbolic temporal discounting, for example – but some things were slightly less expected.
For example, while 39% of my students hand in everything on time, 30% of people who hand in their first assignment late then go on to have a blemish-free future record. However, students who hand up that first assignment late are approximately twice as likely to have problems – which moves this group into a weakly classified at-risk category. Now, I note that this is before any marking has taken place, which means that, if you’re tracking submissions, one very quick and easy way to detect people who might be having problems is to look at the first assignment submission time. This inspection takes about a second and can easily be automated, so it’s a very low burden scheme for picking up people with problems. A personalised response, with constructive feedback or a gentle question, in the zone where the student should have submitted (but didn’t), can be very effective here. You’ll note that I’m working with late submitters not non-submitters. Late submitters are trying to stay engaged but aren’t judging their time or allocating resources well. Non-submitters have decided that effort is no longer worth allocating to this. (One of the things I’m investigating is whether a reminder in the ‘late submission’ area can turn non-submitters into submitters, but this is a long way from any outcomes.)
I should note that the type of assignment work is important here. Computer programs, at least in the assignments that we set, are not just copied in from text. They are not remembering it or demonstrating understanding, they are using the information in new ways to construct solutions to problems. In Bloom’s revised taxonomic terms, this is the “Applying” phase and it requires that the student be sufficiently familiar with the work to be able to understand how to apply it.
I’m not measuring my students’ timeliness in terms of their ability to show up to a lecture and sleep or to hand up an essay of three paragraphs that barely meets my requirements because it’s been Frankenwritten from a variety of sources. The programming task requires them to look at a problem, design a solution, implement it and then demonstrate that it works. Their code won’t even compile (turn into a form that a machine can execute) unless they understand enough about the programming language and the problem, so this is a very useful indication of how well the student is keeping up with the demands of the course. By focusing on an “Applying” task, we require the student to undertake a task that is going to take time and the way in which they assess this resource and decide on its management tells us a lot about their metacognitive skills, how they are situated in the course and, ultimately, how at-risk they actually are.
Looking at assignment submission patterns is a crude measure, unashamedly, but it’s a cheap measure, as well, with a reasonable degree of accuracy. I can determine, with 100% accuracy, if a student is at-risk by waiting until the end of the course to see if they fail. I have accuracy but no utility, or agency, in this model. I can assume everyone is at risk at the start and then have the inevitable problem of people not identifying themselves as being in this area until it’s too late. By identifying a behaviour that can lead to problems, I can use this as part of my feedback to illustrate a concrete issue that the student needs to address. I now have the statistical evidence to back up why I should invest effort into this approach.
Yes, you get a lot of excuses as to why something happened, but I have derived a great deal of value from asking students questions like “Why did you submit this late?” and then, when they give me their excuse, asking them “How are you going to avoid it next time?” I am no longer surprised at the slightly puzzled look on the student’s face as they realise that this is a valid and necessary question – I’m not interested in punishing them, I want them to not make the same mistake again. How can we do that?
I’ll leave the rest of this discussion for after my talk on Monday.
Loading the Dice: Show and Tell
Posted: September 6, 2012 Filed under: Education | Tags: authenticity, curriculum, design, education, educational problem, higher education, in the student's head, learning, principles of design, resources, teaching, teaching approaches, thinking, tools Leave a commentI’ve been using a set of four six-sided dice to generate random numbers for one of my classes this year, generally to establish a presentation order or things like that. We’ve had a number of students getting the same number and so we have to have roll-offs. Now in this case, the most common number rolled so far has been in the range of 17-19 but we have only generated about 18-20 rolls so, while that’s a little high, it’s not high enough to arouse suspicion.
Today we rolled again, and one student wasn’t quite there yet so I did it with the rest of the class. Once again, 18 showed up a bit. This time I asked the class about it. Did that seem suspicious? Then I asked them to look at the dice.
Oh.
Only two of the dice are actually standard dice. One has the number five on every face. One has three sixes and three twos. The students have seen these dice numerous times and have never actually examined them – of course, I didn’t leave them lying around for them to examine but, despite one or two starting to think “Hey, that’s a bit weird”, nobody ever twigged to the loading.

All of the dice in this picture are loaded through weight manipulation, rather than dot alteration. You can buy them for just about any purpose. Ah, Internet!
Having exposed this trick, to some amusement, the last student knocked on the door and I picked up the dice. He was then asked to roll for his position, with the rest of the class staying quiet. (Well, smirking.) He rolled something in 17-19, I forget what, and I wrote that up on the board. Then I asked him if it seemed high to him? On reflection, he said that these numbers all seemed pretty high, especially as the theoretical maximum was 24. I then asked if he’d like to inspect the dice.
He then did so, as I passed him the dice one at a time, and storing the inspected dice in my other hand. (Of course, as he peered at each die to see if it was altered, I quickly swapped one of the ‘real’ dice back into the position in my hand and, as the rest of the class watched and kept admirably quiet, I then forced a real die onto him. Magic is all about misdirection, after all.)
So, having inspected all of them, he was convinced that they were normal. I then plonked them down on the table and asked him to inspect them, to make sure. He lined them up, looked across the top face and, then, looked at the side. Light dawned. Loudly! What, of course, was so startling to him was that he had just inspected the dice and now they weren’t normal.
What was my point?
My students have just completed a project on data visualisation where they provided a static representation of a dataset. There is a main point to present, supported by static analysis and graphs, but the poster is fundamentally explanatory. The only room for exploration is provided by the poster producer and the reader is bound by the inherent limitations in what the producer has made available. Much as with our discussions of fallacies in argument from a recent tutorial, if information is presented poorly or you don’t get enough to go on, you can’t make a good decision.
Enter, the dice.
Because I deliberately kept the students away from them and never made a fuss about them, they assumed that they were normal dice. While the results were high, and suspicion was starting to creep in, I never gave them enough space to explore the dice and discern their true nature. Even today, while handing them to a student to inspect, I controlled the exploration and, by cherry picking and misdirection, managed to convey a false impression.
Now my students are moving into dynamic visualisation and they must prepare for sharing data in a way that can be explored by other people. While the students have a lot of control over who this exploration takes place, they must prepare for people’s inquisitiveness, their desire to assemble evidence and their tendency to want to try everything. They can’t rely upon hiding difficult pieces of data in their representation and they must be ready for users who want to keep exploring through the data in ways that weren’t originally foreseen. Now, in exploratory mode, they must prepare for people who want to try to collect enough evidence to determine if something is true or not, and to be able to interrogate the dataset accordingly.
Now I’m not saying that I believe that their static posters were produced badly, and I did require references to support statements, but the view presented was heavily controlled. They’ve now seen, in a simple analogue, how powerful that can be. Now, it’s time to break out of that mindset and create something that can be freely explored, letting their design guide the user to construct new things rather than to lead them down a particular path.
I can only hope that they’re exceed by this because I certainly am!!
(Reasonable) Argument, Evidence and (Good) Journalism: Is “Crimson” the Colour of Their Faces?
Posted: September 5, 2012 Filed under: Education | Tags: advocacy, authenticity, blogging, community, education, ethics, grand challenge, higher education, in the student's head, learning, reflection, student perspective, teaching, teaching approaches, thinking 2 CommentsI ran across a story on the Harvard Crimson about a surprisingly high level of suspected plagiarism in a course, Government 1310. The story opens up simply enough in the realms of fact, where the professor suspected plagiarism behaviour in 10-20 take home exams, which was against published guidelines, and has now expanded to roughly 125 suspicious final exams. There was a brief discussion of the assessment of the course and the steps taken so far by the faculty.
Then, the article takes a weird turn. Suddenly, we have a student account, an anonymous student who doesn’t wish their name to be associated with the plagiarism, who “suspected that Government 1310 was the course in question”. Hello? Why is this… ahhh. Here’s some more:
Though she said she followed the exam instructions and is not being investigated by the Ad Board, she said she thought the exam format lent itself to improper academic conduct.
“I can understand why it would be very easy to collaborate,” said the student
Oh. Collaborate. Interesting. Next we get the Q Guide rating for the course and this course gets 2.54/5 versus the apparent average of 3.91. Then we get some reviews from the Q Guide that “spoke critically of the course’s organisation and the difficulty of the exam questions”.
Spotting a pattern yet?
Another student said that he/she had joined a group of 15 other students just before the submission date and that they had been up all night trying to understand one of the questions (worth 20%).
I submitted this to my students to read and then asked them how they felt about it. Understandably, by the end of the reading, while my students were still thinking about plagiarism, they were thinking that there may have been some… justification. Then we started pulling the article apart.
When we start to look at the article, it becomes apparent that the facts presented all have a rather definite theme – namely that if cheating has occurred, that it has a justification because of the terrible way the course was taught (low Q Guide rating! 16 students confused!)
Now, I can not see the Q Guide data, because when I go to the page I get this information (and I need a Harvard login to go further):
Q Guide
The Q Guide was an annually published guide that reported the results of each year’s course evaluations. Formerly called the CUE Guide, it was renamed the Q Guide in 2007 because the evaluations now include the GSAS and are no longer run solely by the Committee on Undergraduate Education (CUE). In 2009, in place of The Q Guide, Harvard College integrated Q data with the online course selection tool (at my.harvard.edu), providing a simple and easy way to access and compare course evaluation data while planning your course schedule.
So if the article, regarding an exam run in 2012, is referring to the Q Guide for Gov 1310, then it’s one of two things: using an old name for new data (admittedly, fairly likely) or referring to old data. The question does arise, however, whether the Q Guide rating refers to this offering or a previous offering. I can’t tell you which it is because I don’t know. It’s not publicly available and the article doesn’t tell me. (Although you’ll note that the Q Guide text refers to this year‘s evaluations. There’s a part of me that strongly suspects that this is historical data but, of course, I’m speculating.)
However, the most insidious aspect is the presentation of 16 students who are confused about content in a way that overstates their significance. It’s a blatant example of emotive manipulation and encourages the reader to make a false generalisation. There were 279 students enrolled in Gov 1310. 16 is 5.7%. Would I be surprised in somewhere around 5% of my students weren’t capable of understanding all of the questions or thought that some material wasn’t in the course?
No, of course not. That’s roughly the percentage of my students who sometimes don’t know which Dr Falkner is teaching their class. (Hint: one is male and one is female. Noticeably so in both cases.)
I presented this to my Grand Challenge students as part of our studies of philosophical and logical fallacies, discussing how arguments are made to mislead and misdirect. The terrible shame is that, with a detected rate of plagiarism that is this high, I would usually have a very detailed look at the learning and teaching strategies employed (how often are exams being rewritten, how is information being presented, how is teaching being carried out) because this is an amazingly high level of suspected plagiarism.
Despite the misleading journalism presented in the Crimson, the course and its teachers may have to shoulder some responsibility here. As always, just because someone’s argument is badly made, doesn’t mean that it is actually wrong. It’s just disappointing that such a cheap and emotive argument was raised in a way that further fogs an important issue.
As I said to my students today, one of the most interesting way to try to understand a biassed or miscast argument is to understand who the bias favours – cui bono? (To whom the benefit? I am somewhat terrified, on looking for images for this phrase, that it has been highjacked by extremists and conspiracy theorists. It’s a shame because it’s historically beautiful.)
So why would the Crimson run this? It’s pretty manipulative so, unless this is just bad journalism, cui bono?
Having looked up how disciplinary boards are constituted at Harvard, I found a reference that there are three appointed faculty members and:
There are three students appointed to the board as full voting members. Two of these will be assigned to specific cases on a case-by-case basis and will not be in the same division as the student facing disciplinary action.
In this case, the Crimson’s story suddenly looks a lot… darker. If, by publishing this article, they reach the right students and convince them the action of the suspected plagiarists may have been overly influenced by academics who are not performing their duties – then we risk suddenly having a deadlocked board and a deleterious effect on what should have been an untainted process.
The Crimson has further distinguished itself with a follow-up article regarding the uncertainty students are feeling because of the process.
“It’s unfair to leave that uncertainty, given that we’re starting lives,” said the alumnus, who was granted anonymity by The Crimson because he said he feared repercussions from Harvard for discussing the case.
Oh, Harvard, you giant monster, unfairly delaying your decision on a plagiarism case because the lecturers were so very, very bad that students had to cheat. And, what’s worse, you are so evil that students are scared of you – they “fear the repercussions”!
Thank you, Crimson, for providing so much rich fodder for my discussion on how the words “logical argument”, “evidence” and “good journalism” can be so hard to fit into the same sentence.
The Precipice of “Everything’s Late”
Posted: September 3, 2012 Filed under: Education | Tags: advocacy, authenticity, education, feedback, higher education, learning, measurement, research, resources, student perspective, teaching, teaching approaches, thinking, time banking 7 CommentsI spent most of today working on the paper that I alluded to earlier where, after over a year of trying to work on it, I hadn’t made any progress. Having finally managed to dig myself out of the pit I was in, I had the mental and timeline capacity to sit down for the 6 hours it required and go through it all.
In thinking about procrastination, you have to take into account something important: the fact that most of us work in a hyperbolic model where we expend no effort until the deadline is right upon us and then we put everything in, this is temporal discounting. Essentially we place less importance on things in the future than the things that are important to us now. For complex, multi-stage tasks over some time this is an exceedingly bad strategy, especially if we focus on the deadline of delivery, rather than the starting point. If we underestimate the time it requires and we construct our ‘panic now’ strategy based on our proximity to the deadline, then we are at serious risk of missing the starting point because, when it arrives, it just won’t be that important.
Now, let’s increase the difficulty of the whole thing and remember that the more things we have to think about in the present, the greater the risk that we’re going to exceed our capacity for cognitive load and hit the ‘helmet fire’ point – we will be unable to do anything because we’ve run out of the ability to choose what to do effectively. Of course, because we suffer from a hyperbolic discounting problem, we might do things now that are easy to do (because we can see both the beginning and end points inside our window of visibility) and this runs the risk that the things we leave to do later are far more complicated.
This is one of the nastiest implications of poor time management: you might actually not be procrastinating in terms of doing nothing, you might be working constantly but doing the wrong things. Combine this with the pressures of life, the influence of mood and mental state, and we have a pit that can open very wide – and you disappear into it wondering what happened because you thought you were doing so much!
This is a terrible problem for students because, let’s be honest, in your teens there are a lot of important things that are not quite assignments or studying for exams. (Hey, it’s true later too, we just have to pretend to be grownups.) Some of my students are absolutely flat out with activities, a lot of which are actually quite useful, but because they haven’t worked out which ones have to be done now they do the ones that can be done now – the pit opens and looms.
One of the big advantages of reviewing large tasks to break them into components is that you start to see how many ‘time units’ have to be carried out in order to reach your goal. Putting it into any kind of tracking system (even if it’s as simple as an Excel spreadsheet), allows you to see it compared to other things: it reduces the effect of temporal discounting.
When I first put in everything that I had to do as appointments in my calendar, I assumed that I had made a mistake because I had run out of time in the week and was, in some cases, triple booked, even after I spilled over to weekends. This wasn’t a mistake in assembling the calendar, this was an indication that I’d overcommitted and, over the past few months, I’ve been streamlining down so that my worst week still has a few hours free. (Yeah, yeah, not perfect, but there you go.) However, there was this little problem that anything that had been pushed into the late queue got later and later – the whole ‘deal with it soon’ became ‘deal with it now’ or ‘I should have dealt with that by now’.
Like students, my overcommitment wasn’t an obvious “Yes, I want to work too hard” commitment, it snuck in as bits and pieces. A commitment here, a commitment there, a ‘yes’, a ‘sure, I can do that’, and because you sometimes have to make decisions on the fly, you suddenly look around and think “What happened”? The last thing I want to do here is lecture, I want to understand how I can take my experience, learn from it, and pass something useful on. The basic message is that we all work very hard and sometimes don’t make the best decisions. For me, the challenge is now, knowing this, how can I construct something that tries and defeats this self-destructive behaviour in my students?
This week marks the time where I hope to have cleared everything on the ‘now/by now’ queue and finally be ahead. My friends know that I’ve said that a lot this year but it’s hard to read and think in the area of time management without learning something. (Some people might argue but I don’t write here to tell you that I have everything sorted, I write here to think and hopefully pass something on through the processes I’m going through.)




