Group feedback, fast feedback, good feedback

We had the “first cut” poster presentation today in my new course. Having had the students present their pitches the previous week, this week was the time to show the first layout – put up your poster and let it speak for itself.

The results were, not all that surprisingly, very, very good. Everyone had something to show, a data story to tell and some images and graphs that told the story. What was most beneficial though was the open feedback environment, where everyone learned something from the comments on their presentation. One of my students, who had barely slept for days and was highly stressed, got some really useful advice that has given him a great way forward – and the ability to go to bed tonight with the knowledge that he has a good path forward for the next two weeks.

Working as a group, we could agree as a group, discuss and disagree, suggest, counter-suggest, develop and enhance. My role in all of this is partially as a ‘semi-expert’ but also as a facilitator. Keep the whole thing moving, keep it to time, make sure that everyone gets a good opportunity to show their work and give and receive feedback.

The students all write down their key feedback, which is scanned as a whole and put on the website so that any good points that went to anyone can now be used by anyone in the group. The feedback is timely, personal and relevant. Everyone feels that these sessions are useful and the work produced reflects the advantages. But everyone talks to everyone else – it’s compulsory. Come to the session, listen and then share your thoughts.

This, of course, reveals one of my key design approaches: collaboration is ok and there is no competitiveness. Read anything about the grand challenges and you keep seeing the word ‘community’ through it. Solid and open communities, where real and effective sharing happens, aren’t formed in highly competitive spaces. Because the students have unique projects, they can share ideas, references and even analysis techniques without plagiarism worries – because they can attribute without the risk of copying. Because there is no curve grading, helping someone else isn’t holding you back.

Because of this, we have already had two informal workshop groups form to address issues of analysis and software, where knowledge passes from person to person. Before today’s first cut presentation, a group was sitting outside, making suggestions and helping each other out – to achieve some excellent first cut results.

Yes, it’s a small group so, being me, now I’m worrying about how I would scale this up, how I would take this out to a large first-year class, how I would get it to a school group. This groups need careful facilitation and the benefit of inter-group communication is derived from everyone in the group having a voice. The number of interactions scale with the square of the group size, so there’s a finite limit to how many people I can have in the group and fit it into a two-hour practical session. If I split a larger class into sub-groups, I lose the advantage of everyone see in everyone else’s work.

But this can be solved, potentially with modern “e-” techniques, or a different approach to preparation, although I can’t quite see it yet. There’s a part of me that thinks “Ask these students how they would approach it”, because they have viewpoints and experience in this which complements mine.

Every week that goes by, I wonder if we will keep improving, and keep rewarding the (to be honest) risk that we’re taking in running a small course like this in leaner times. And, every week, the answer is a resounding “yes”!

Here’s to next week!


Putting it all together – discussing curriculum with students

One of the nice things about my new grand challenges course is that the lecture slots are a pre-reading based discussion of the grand challenges in my discipline (Computer Science), based on the National Science Foundation’s Taskforce report. Talking through this with students allows us to identify the strengths of the document and, perhaps more interestingly, some of its shortfalls. For example, there is much discussion on inter-disciplinary and international collaboration as being vital, followed by statements along the lines of “We must regain the ascendancy in the discipline that we invented!” because the NSF is, first and foremost, a US-funded organisation. There’s talk about providing the funds for sustainability and then identifying the NSF as the organisation giving the money, and hence calling the shots.

The areas of challenge are clearly laid out, as are the often conflicting issues surrounding the administration of these kinds of initiative. Too often, we see people talking about some amazing international initiative – only to see it fail because nobody wants to go first, or no country/government wants to put money up that other people can draw on until everyone does it at the same time.

In essence, this is a timing and trust problem. If we may quote Wimpy from the Popeye cartoons:

A picture of Wimpy saying "I will gladly pay you Tuesday for a hamburger today!"

Via theawl.com. Click on the link for a very long discussion of Popeye and Wimpy related issues.

The NSF document lays bare the problem we always have: those who have the hamburgers are happy to talk about sharing the meal but there are bills to be paid. The person who owns the hamburger stand is going to have words with you if you give everything away with nothing to show in return except a promise of payment on Tuesday.

Having covered what the NSF considered important in terms of preparing us for the heavily computerised and computational future, my students finished with a discussion of educational issues and virtual organisations. The educational issues were extremely interesting because, having looked at the NSF Taskforce report, we then looked at the ACM/IEEE 2013  Computer Science Strawman curriculum to see how many areas overlapped with the task force report. Then we looked at the current curriculum of our school, which is undergoing review at the moment but was last updated for the 2008 ACM/IEEE Curriculum.

What was pleasing was, rom the range of students, how many of the areas were being addressed throughout our course and how much overlap there was between the highlighted areas of the NSF Report and the Strawman. However, one of the key issues from the task force report was the notion of greater depth and breadth – an incredible challenge in the time-constrained curriculum implementations of the 21st century. Adding a new Knowledge Area (KA) to the Strawman of ‘Platform Dependant Computing’ reflects the rise of the embedded and mobile device yet, as the Strawman authors immediately admit, we start to make it harder and harder to fit everything into one course. Combine this with the NSF requirement for greater breadth, including scientific and mathematical aspects that have traditionally been outside of Computing, and their parallel requirement for the development of depth… and it’s not easy.

The lecture slot where we discussed this had no specific outcomes associated with it – it was a place to discuss the issues arising but also to explain to the students why their curriculum looks the way that it does. Yes, we’d love to bring in Aspect X but where does it fit? My GC students were looking at the Ethics aspects of the Strawman and wondered if we could fit Ethics into its own 3-unit course. (I suspect that’s at least partially my influence although I certainly didn’t suggest anything along these lines.) “That’s fine,” I said, “But what do we lose?”

In my discussions with these students, they’ve identified one of the core reasons that we changed teaching languages, but I’ve also been able to talk to them about how we think as we construct courses – they’ve also started to see the many drivers that we consider, which I believe helps them in working out how to give feedback that is the most useful form for us to turn their needs and wants into improvements or developments in the course. I don’t expect the students to understand the details and practice of pedagogy but, unless I given them a good framework, it’s going to be hard for them to communicate with me in a way that leads most directly to an improved result for both of us.

I’ve really enjoyed this process of discussion and it’s been highly rewarding, again I hope for both sides of the group, to be able to discuss things without the usual level of reactive and (often) selfish thinking that characterises these exchanges. I hope this means that we’re on the right track for this course and this program.


Partnership vs Prison Experiment

The Stanford Prison Experiment (official site, Wikipedia, BBC Recreation)  is notorious in many ways. For those who haven’t heard of it, in 1971, a randomly selected group of 24 males (out of 70) were split into two randomly assigned groups: prisoners and guards. They were then placed into a mock prison situation. Despite agreeing to a 7-14 day experimental run, the experiment was terminated after 6 days. By this stage, 1/3 of the guards were showing sadistic tendencies, 2 prisoners had quit and the abuse that prisoners were suffering included solitary confinement, loss of mattresses, reduced access to toilets (or enforced primitive access).

Lest you think that the researcher controlling it, Professor Philip Zimbardo, terminated this for altruistic reasons, it was in response to the objections of a graduate student who was observing the experiment – who he was dating and went on to marry. Of the fifty people who had observed the experiment, and the deteriorating conditions, Zimbardo claims that only this one observer objected.

I assume that some of you thinking “Surely… someone else said something?” Yeah, I thought that too, when I first read about it. Apparently not.) It’s worth noting that Zimbardo’s prison started out as a more ‘extreme’ prison than usual, with degrading activities forced onto the prisoners fairly early. You can read about these in the site or you can read about the Abu Ghraib incident, which is strikingly similar.

What is worth noting up front is that this research has never been fully successfully replicated, for a number of reasons, and the publication standard was low. The Stanford Prison Experiment stands, however, in many ways as a failure to protect the people in an experiment, even if the actions of the agents in the system was not as random (there are claims Zimbardo engineered large sections of this) or as meaningful (the experiment was very poorly constructed) as it may appear.

The random selection of the participants, 24 out of an original 70 and randomised roles, seems to indicate a situational attribution of behaviour, rather than one that we are born with. Put someone in a position where they have power over somebody else, put enough rules in the way – *bang* you’re potentially recreating the Stanford Prison Experiment. (Paging Dr Milgram… a topic for a later post.)

Ultimately, demanding compliance can place us in difficult positions where we require authoritarianism on the part of those who demand, and compliance from those who must obey. Whatever the Stanford study showed, this arrangement of struct power-divided rules does not allow for a meeting of minds in any kind of partnership.

One of things I dislike the most about some of the seemingly arbitrary things that we sometimes do (or are encouraged to do) in teaching is that any situation that devolves to “because I said so” or “because I’ve been told to” is requiring compliance under the aegis of institutional support, and driven by some legitimising framework. This gets in the way of one of the most useful and constructive relationships that we can form – the partnership between educator and student. Now, I’m not suggesting for a second that most students have the maturity or depth of knowledge to devise and run entire courses but a partnership role allows us to avoid falling into the traps of guard and prisoner. We do have hard limits that we need to adhere to, to make the recognition of education possible in many senses, but building courses that clearly set these limits in a constructive and useful way, rather than a reactive and inauthentic way, pulls us out of the “I told you to do this” and allows us to move into the “why didn’t you do that?”

(We could talk about allowing individuals mobility to reduce their dependency on external validation from their peers, and hence allow us to encourage the pursuit of individual goals and reduce any fighting over favouritism but I’m not well-versed enough in social identity theory yet to give this much flesh.)

I, as the subject matter expert, am trying to assist the student in developing knowledge within a particular set of subjects and any useful associated areas. If I have created something where, in order to understand the work, you need to complete certain readings and assignments, perform certain actions, and do so in a certain timeframe or lose the opportunity to participate – most students will actually do this. On top of the issues of knowledge, we have the other skills that we are trying to transfer: design, time management, ethics, professionalism, communication skills. This is where it gets hard.

Say, for example, I design a course where you need to finish Assignment 2 before we discuss a certain topic in a lecture/tutorial/studio activity. Therefore, you have a reason to finish assignment 2 before some deadline. I can set a deadline that is just before the next activity or I can set it a few days before to give people some digestion time prior to looking at it again. Or I can set an earlier deadline to give people practice at time management. However, if Assignment 2 is work that will not be referred to elsewhere in the course, except for the exam, when should I set the deadline?

The problem we have is that allowing deadlines to run late means no marking or feedback until late – this, of course, drives our education design to bring formative work forward but, once again, this only makes sense if that feedback will be useful earlier on.

So, to briefly recap, setting an arbitrary hand-in time that is purely to make your marking life easier and has no pedagogical driver or no impact on student learning is understandable but, in many ways, potentially an abuse of your position. (I am all too familiar of the realities of staff and resource shortages on when and how we can mark, especially when we start getting told to increase feedback or have all assignments back within time X. But let’s get this straight: formative and summative have different roles and marking loads. We know that we can achieve things with good learning design that far exceed what we can manage with arbitrary action.)

Now let’s look at a more complex issue – late penalties. I have evidence that students change their behaviour when late penalties are fixed on 24 hour barriers. We’ve seen students line up with these and start handing up in response to these new barriers: miss one and you lose even more marks. But have we changed the right behaviour or does this merely lead to a certain form of resignation in the face of arbitrary authority?

Why am I removing marks anyway? If the work is handed in before the time that it’s needed, then, from a knowledge point of view, the aim has been achieved. Which skill am I developing? If you responded with ‘time management’, then providing that we are completely clear on when the work must be handed in to achieve certain requirements AND that we have added an overall factor in the ‘professional’ spectrum of time management, we are probably doing the right thing. If we’re just saying “hand it in on time OR ELSE” then we are conflating issues of knowledge development with issues of compliance and this is where it starts to get murky.

Now it doesn’t have to get murky but it’s completely possible in this zone. You risk ending up academics who won’t accept anything because it’s late (regardless of reason) or students who start acting up (out of defiance) or, potentially worse, students who become completely passive and dependent upon your authority. If self-regulation is supposed to be in play, then we haven’t achieved much by doing this.

Nothing I’ve said should be interpreted as “no deadlines” or “no authority” but what I am saying is that we know what happens when we take a randomly assigned group of people and make one beholden to the other, when there is no really good reason or sense of equality or partnership between them. We’ve seen it time and time again.

Kohn, in “Punishing with Rewards”, makes a number of observations, some good and some bad, including that one of our biggest risks is in the rupturing of relationships by setting up a disparity of power levels, where one person controls and the other person complies or seeks to appease, rather than to achieve the actual objective. It’s an interesting way to look at a very challenging problem, to give us more lines along which to think.

I should finish this by noting, again, that Zimbardo’s experiment was flawed in many ways and deriving significance from the role is hard. It appears, from the UK version, that leadership plays a key part in what happens. It was only when strong leadership started to lead the prison guards down dark paths in the UK recreation that they started to approach what had happened in Stanford. Zimbardo admits that his role in the experiment may have been not been all that sensible in many ways but it may be that his briefing set the scene for what happened. His passive observation as matters deteriorated, with the guards knowing that he was watching, certainly validated their actions. Either way, if it is a fact that one key leader can have so much impact, then that makes what we do even more important – even if it’s occasionally looking at something, thinking about it and saying ‘No, actually, that’s wrong.”


The Key Difference (or so it appears): Do You Love Teaching?

Please, any of my students reading this, do not give me an apple. I have sufficient!

I wander around fair bit for work. (I make it sound more impressive than that but the truth is that I end up in lots of different places to work on my many projects and sometimes the movement, although purposeful, is more Brownian than not – due to life.) I’ve had a chance to talk to a lot of people who teach – some of whom are putting vast amounts of effort into it and some of whom aren’t.

The key difference, unsurprisingly, is generally the passion behind it. We see this in our students. They will spend days working on a Minecraft construction to simulate an Arithmetic and Logic Unit, but won’t always put in the two hours to write 20 lines of C++ code. They will write 20,000 words on their blog but can’t give you a 1,000 word summary.

We put effort into the things that we are interested in. Sure, if we’re really responsible and have self-regulation nailed, then we can do things that we’re not interested in, or actively dislike, but it’s never really going to have the same level of effort or commitment.

Passion (or the love of something) is crucial. Some days I have so much to say on the blog that I end up with 4-5 days stocked up in the queue. Some days I struggle to come up with the daily post or, as yesterday, I just run out of time to hit the 4am post cycle because I am doing other things that I am passionate about. Today, of course, the actual deadline timer is running and it seems to have made me think – now I’m passionate and now you’ll get something worth reading. If I’d stayed up until after my guests had left last night, written just anything to meet the deadline? It wouldn’t be anywhere near my best work.

Passion is crucial.

Which brings me to teaching. I know a lot of academics – some who are research/teaching/admin, some research only, some teaching/admin and… well, you get the picture. The majority are the ‘3-in-1’ academics and, in many regards, looking at their student evaluations and performance metrics will not tell you anything about them as a teacher that you can’t learn by sitting down with them and talking about their teaching. It is hard to shut me up about my courses and my students, the things I’m trying, the things I’m thinking of adopting, the other areas I’m looking at, the impact of what other Unis and people are doing, the impact of reports. I am a (junior) scholar in the discipline of learning and teaching and I really, really  love teaching. For me, putting effort into it is inevitable, to a great extent.

Then I talk to colleagues who really just want to do their research and be left alone. Everything else is a drag on their research. Administration will get the minimum effort, if it’s done. Teaching is something that you have to do and, if the students don’t get it, then it’s their fault. What is so weird about this is that these people are, in the vast majority, excellent scholars in their own discipline. They research and read heavily, they are aware of what every other researcher is doing in this area, they know if their work has a chance for publication or grants. Having these skills, they then divide the world into ‘places where I have to scholarly’ and ‘places where I can phone it in’. (Not all researchers are like this, I’m talking about the ones who consider anything other than research beneath them.)

What a shame! What a terrible missed opportunity for both these people who should be more aware of the issues of learning and teaching, and for the students who could be learning so much more from them? But when you actually talk to these academics, some of them just don’t liked teaching, they don’t see the point of putting effort into it or (in some cases) they just don’t know what to do and how to improve so they hunker down and try to let it all slide around them.

Part of this is the selective memory that we have of ourselves as students. I’m lucky – I was terrible. I was fortunate enough to be aware and mature enough as I reconstructed myself as a good student to see the transformative process in action. A lot of my peers are happy to apply rules to students that they wouldn’t (or don’t ) apply to themselves now or in the past, such as:

“I’m an academic who doesn’t like teaching, despite being told that it’s part of my job, so I’ll do the minimum required – or less on some occasions. You, however, are a student who doesn’t like the sub-standard learning experiences that my indifference brings you but I’m telling you to do it, so just do it or I’ll fail you.”

This isn’t just asymmetrical, this is bordering on the Stanford Prison Experiment, an arbitrary assignation of roles that leads to destructive power-derived behaviour. But, if course, if you don’t enjoy doing something then there are going to be issues.

Have we actually ever asked people these key questions as a general investigation? “Do you like teaching?” “What do you enjoy about teaching?” “What can we do to make you enjoy teaching more?” Would this muddy the water or clear the air? Would this earth our non-teaching teachers and fire them up?

Even where people run vanity courses (very small scale, research-focused courses design to cherry pick the good students) they are still often disappointed because, even where you can muster the passion to teach, if you don’t really understand how to teach or what you need to do to build a good learning experience, then you end up with these ‘good’ students in this ‘enjoyable’ course failing, complaining, dropping out and, in more analogous terms, kicking your puppy. You will now like teaching even less!

It’s blindingly obvious that some people don’t like teaching but, much as we wouldn’t stand out the front of a class and yell “PASS, IDIOTS!”, I’m looking for other good examples where we start to ask people why they don’t want to do it, what they’re worried about, why they don’t respect it and how we can get them more involved in the L&T community.

Let’s face it, when you love teaching, the worst day with the students is still a pretty good day. It would be nice to share this joy further.


In A Student’s Head – Mine From 26 Years Ago

I attended an Australian Council of Deans of ICT Learning and Teaching Academy event run by Elena Sitnikova from the University of South Australia. Elena is one of the (my fellow?) Fellows in ALTA and works in Cyber Security and Forensic Computing. Today’s focus was on discussing the issues in ICT education in Australia, based on the many surveys that have been run, presenting some early work on learning and teaching grants and providing workshops on “Improving learning and teaching practice in ICT Education” and “Developing Teamwork that Works!”. The day was great (with lots of familiar faces presenting a range of interesting topics) and the first workshop was run by Sue Wright, graduate school in Education, University of Melbourne. This, as always, was a highly rewarding event because Sue forced me to go back and think about myself as a student.

This is a very powerful technique and I’m going to outline it here, for those who haven’t done it for a while. Drawing on Bordieu’s ideas on social and cultural capital, Sue asked us to list our non-financial social assets and disadvantages when we first came to University. This included things like:

  • Access to resources
  • Physical appearance
  • Educational background
  • Life experiences
  • Intellect and orientation to study
  • Group membership
  • Accent
  • Anything else!

When you think about yourself in this way, you suddenly have to think about not only what you had, but what you didn’t have. What helped you stay in class?What meant that you didn’t show up? From a personal perspective, I had good friends and a great tan but I had very little life experience, a very poor study ethic, no real sense of consequences and a very poor support network in an academic sense. It really brought home how lucky I was to have a group of friends that kept me coming to University. Of course, in those pre-on-line days, you had to come to Uni to see your friends, so that was a good reason to keep people on campus – it allowed for you to learn things by bumping into a people, which I like to refer to as “Brownian Communication”.

“Hey, Neutrino 3 is trying to put together a Learning and Teaching Grant. Who’s in?”

This exercise made me think about my transition to being a successful student. In my case, it took more than one degree and a great deal more life experience before I was ready to come back and actually succeed. To be honest, if you looked at my base degree, you’d never have thought that I would make it all the way to a PhD and, yet, here I am, on a path where I am making a solid and positive difference.

Sue then reminded people of Hofstede’s work on cultural dimensions – power distance, individualism versus collectivism, and uncertainty avoidance. How do students work – do they need a large ‘respect gap’ between student and teacher? Do they put family before their own study? Do they do anything rather than explore the uncertain? It’s always worth remembering that, where “the other” exists for us, we exist as “the other” reciprocally. While it’s comfortable as white, culturally English and English speaking people to assume that “the other” is transgressing with respect to our ‘dominant’ culture, we may be asking people to do something that is incredibly uncomfortable and goes far beyond learning another language.

One of the workshop participants was born and grew up in Korea and he made the observation that, when he was growing up, the teacher was held at the same level of the King and your father – and you don’t question the King or your father! He also noted that, on occasion, ‘respect’ had to be directed towards teachers that they did not actually respect. He had one bad teacher and, in that class, the students asked no questions and just let the teacher talk. As someone who works within a very small power distance relationship with y students, I have almost never felt disrespected by anything that my students do, unless they are actively trying to be rude and disrespectful. If I have nobody following, or asking questions, then I always start to wonder if I’ve been tuned out and they are listening to the music in their heads. (Or on their iPhones, as it is the 21st Century!)

Australia is a low power distance/high individualism culture with a focus on the short-term in many respects (as evidence by profit and loss quarterly focus and, to be frank, recent political developments). Bringing people from a high PD/high collectivism culture, such as some of those found in South East Asia, will need some sort of management to ensure that we don’t accidentally split the class. It’s not enough to just say “These students do X” because we know that we can, with the right approach, integrate our student body. But it does take work.

As always, working with Sue (you never just listen to Sue, she always gets you working) was a very rewarding and reflective activity. I spent 20 minutes trying to learn enough about a colleague from UniSA, Sang, that I could answer questions about his life. While I was doing this, he was trying to become Nick. What emerged from this was how amazingly similar we actually are – different Unis, different degrees, different focus, one Anglo-origin, one Korean-origin – and it took us quite a while to find things where we were really so different that we could talk about the challenges if we had to take on each other’s lives.

It was great to see most of the Fellows again but, looking around a large room that wasn’t full to the brim, it reminded me that we are often talking to those people who already believe that what we’re doing is the right thing. The people that really needed to be here were the people who weren’t in the room.

I’m still thinking about how we can continue our work to reach out and bring more people into this very, very rewarding community.

 


Silk Purses and Pig’s Ears

There’s an old saying “You can’t make a silk purse out of a pig’s (or sow’s) ear”. It’s the old chestnut that you can’t make something good out of something bad and, when you’re talking about bad grapes or rotten wood, then it has some validity (but even then, not much, as I’ll note later). When it’s applied to people, for any of a large range of reasons, it tends to become an excuse to give up on people or a reason why a lack of success on somebody’s part cannot be traced back to you.

I’m doing a lot of reading in the medical and general ethics as part of my preparation for one of the Grand Challenge lectures. The usual names and experiments show up, of course, when you start looking at questionable or non-existent ethics: Milgram, the Nazis, Stanford Prison Experiment, Unit 731, Tuskegee Syphilis Experiment, Little Albert and David Reimer. What starts to come through from this study is that, in many of these cases, the people being experimented upon have reached a point in the experimenter’s eyes where they are not people, but merely ‘subjects’ – and all too often in the feudal sense as serfs, without rights or ability to challenge what is happening.

But even where the intention is, ostensibly, therapeutic, there is always the question of who is at fault when a therapeutic procedure fails to succeed. In the case of surgical malpractice or negligence, the cause is clear – the surgeon or a member of her or his team at some point made a poor decision or acted incorrectly and thus the fault lies with them. I have been reading up on early psychiatric techniques, as these are full of stories of questionable approaches that have been later discredited, and it is interesting in how easy it is for some practitioners to wash their hands of their subject because they had a lack of “good previous personality” – you can’t make a silk purse out of a pig’s ear. In many cases, with this damning judgement, people with psychiatric problems would often be shunted off to the wards of mental hospitals.

I refer, in this case, to William Sargant (1907-1988), a British psychiatrist who had an ‘evangelical zeal’ for psychosurgery, deep sleep treatment, electroconvulsive therapy (ECT) and insulin shock therapy. Sargant used narcosis extensively, drug induced deep sleep, as he could then carry out a range of procedures on the semi- and unconscious patients that they would have possibly learned to dread if they have received them while conscious. Sargant believed that anyone with psychological problems should be treated early and intensively with all available methods and, where possible, all these methods should be combined and applied as necessary. I am not a psychiatrist and I leave it to the psychiatric and psychotherapy community to assess the efficacy and suitability of Sargant’s methods (they disavow them, for the most part, for what it’s worth) but I mention him here because he did not regard failures as being his fault. It is his words that I am quoting in the previous paragraph. People for whom his radical, often discredited, zealous and occasionally lethal experimentation did not work were their own problem because they lacked a “good previous personality”. You cannot, as he was often quoted to have said, make a silk purse out of a pig’s ear.

How often I have heard similar ideas being expressed within the halls of academia and the corridors of schools. How easy a thing it is to say. Well, one might say, we’ve done all that we can with this particular pupil, but… They’re just not very bright. They daydream in class rather than filling out their worksheets. They sleep at their desks. They never do the reading. They show up too late. They won’t hang around after class. They ask too many questions. They don’t ask enough questions. They won’t use a pencil. They only use a pencil. They talk back. They don’t talk. They think they’re so special. Their kind never amounts to anything. They’re just like their parents. They’re just like the rest of them.

“We’ve done all we can but you can’t make a silk purse out of a sow’s ear.”

As always, we can look at each and every one of those problems and ask “Why?” and, maybe, we’ll get an answer that we can do something about. I realise that resources and time are both scarce commodities but, even if we can’t offer these students the pastoral care that they need (and most of those issues listed above are more likely to be social/behavioural than academic anyway), let us stop pretending that we can walk away, blameless, as Sargant did because these students are fundamentally unsalvageable.

Yeah, sorry, I know that I go on about this but it’s really important to keep on hammering away at this point, every time that I see how my own students could be exposed to it. They need to know that the man that they’re working with expects them to do things but that he understands how much of his job is turning complex things into knowledge forms that they can work with – even if all he does is start the process and then he hands it to them to finish.

Do you want to know how to make great wine? Start with really, really good grapes and then don’t mess it up. Want to know how to make good wine? Well, as someone who used to be a reasonable wine maker, you can give me just about anything – good fruit, ok fruit, bad fruit, mouldy fruit – and I could turn it into wine that you would happily drink. I hasten to point out that I worked for good wineries and the vast quantity of what I did was making good wine from good grapes, but there were always the moments where you had something that, from someone else’s lack of care or inattention, had got into a difficult spot. Understanding the chemical processes, the nature of wine and working out how we could recover  the wine? That is a challenge. It’s time consuming, it takes effort, it takes a great deal of scholarly knowledge and you have to try things to see if they work.

In the case of wine, while I could produce perfectly reasonable wine from bad grapes, simple chemistry prevents me from leaving in enough of the components that could make a wine great. That is because wine recovery is all about taking bad things out. I see our challenge in education as very different. When we find someone who is need of our help, it is what we can put in that changes them. Because we are adding, mentoring, assisting and developing, we are not under the same restrictions as we are with wine – starting from anywhere, I should be able to help someone to become a great someone.

The pig’s ears are safe because I think that we can make silk purses out of just about anything that we set our minds to.


Grand Challenges Course: Great (early) progress on the project work.

While I’ve been talking about the project work in my new “Grand Challenge”-based course a lot, I’ve also identified a degree of, for want of a better word, fearfulness on the part of the students. Given that their first project is a large poster with a visualisation of some interesting data, which they have to locate and analyse, and that these are mostly Computer Science students with no visualisation experience, they are understandably slightly concerned. We’ve been having great discussions and lots of contributions but next week is their first pitch and, suddenly, they need a project theme.

I’ve provided a fair bit of guidance for the project pitch, and I reproduce it here in case you’re interested:

Project 1: First Deliverable, the Pitch

Due 2pm, Wednesday, the 8th of August Because group feedback is such an important part of this project, you must have your pitch ready to present for this session and have the best pitch ready that you can. Allocate at least 10 hours to give you enough time to do a good job.

What is the pitch?

A pitch is generally an introduction of a product or service to an audience who knows nothing about it but is often used to expand knowledge and provide a detailed description of something that the audience is already partially familiar with. The key idea is that you wish to engage your audience and convince them that what you are proposing is worth pursuing. In film-making, it’s used to convey an idea to people who need to agree to support it from a financial or authority perspective.

One of the most successful pitches in Hollywood history is (reputedly) the four word pitch used to convince a studio to fund the movie “Twins”. The pitch was “Schwarzenegger. De Vito. Twins.”

You are not trying to sell anything but you are trying to familiarise a group of people with your project idea and communicate enough information that the group can give you useful feedback to improve your project. You need to think carefully about how you will do this and I strongly suggest that you rehearse before presenting. Trust me when I say that very few people are any good at presentation without rehearsal and I will generally be able to tell the amount of effort that you’ve expended. An indifferent presentation says that you don’t care – and then you have to ask why anyone else would be that motivated to help you.

If you like the way I lecture, then you should know that I still rehearse and practice regularly, despite having been teaching for over 20 years.

How will it work?

You will have 10 minutes to present your project outline. During this time you will:

  • Identify, in one short and concise sentence, what your poster is about.
  • Clearly state the purpose.
  • Identify your data source.
  • Answer all of the key questions raised in the tutorial.
  • Identify your starting strategy, based on the tools given in the tutorial, with a rough outline of a timeline.
  • Outline your analysis methodology.
  • Summarise the benefits of this selection of data and presentation – why is it important/useful?
  • Show a rough prototype layout on an A3 format.

We will then take up to 10 minutes to provide you with constructive feedback regarding any of these aspects. Participants will be assessed both on the pitch that they present and the quality of their feedback and critique. Critique guides will be available for this session.

How do I present it?

This is up to you but I would suggest that you summarise the first seven points as a handout, and provide a copy of your A3 sketch, for reference during critique. You may also use presentations (PowerPoint, Keynote or PDF) if you wish, or the whiteboard. As a guideline, I would suggest no more than four slides, not including title, or your poster sketch. You may use paper and just sketch on that – the idea and your ability to communicate it are paramount at this stage, not the artfulness of the rough sketch.

Important Notes

Some people haven’t been getting all of their work ready on time and, up until now, this has had no impact on your marks or your ability to continue working with the group. If you don’t have your project ready, then I cannot give you any marks for your project and you miss out on the opportunity for group critique and response – this will significantly reduce your maximum possible mark for this project.

I am interested in you presenting something that you find interesting or that you feel will benefit from working with – or that you think is important. The entire point of this course is to give you the chance to do something that is genuinely interesting and to challenge yourself. Please think carefully about your data and your approach and make sure that you give yourself the opportunity to make something that you’d be happy to show other people, as a reflection of yourself, your work and what you are capable of.

END OF THE PITCH DESCRIPTION

We then had a session where we discussed ideas, looked at sources and started to think about how we could get some ideas to build a pitch on. I used small group formation and a bit of role switching and, completely unsurprisingly to the rest of you social constructivists, not only did we gain benefit from the group work but it started to head towards a self-sustaining activity. We went from “I’m not really sure what to do” to something very close to “flow” for the majority of the class. To me it was obvious that the major benefit was that the ice had been broken and, through careful identification of what to happen with the ideas and a deliberate use of Snow’s Cholera diagram as an example of how powerful a good (but fundamentally) simple visualisation could be, the group was much better primed to work on the activity.

The acid test will be next week but, right now, I’m a lot more confident that I will get a good set of first pitches. Given how much I was holding my breath, without realising it, that’s quite a good thing!


The 1-Year Degree – what’s your reaction?

I’m going to pose a question and I’d be interested in your reaction.

“Is there a structure and delivery mechanism that could produce a competent professional graduate from a degree course such as engineering or computer science, which takes place over a maximum of 12 months including all assessment, without sacrificing quality or content?”

What was your reaction? More importantly, what is the reasoning behind your reaction?

For what it’s worth my answer is “Not with our current structures but, apart from that, maybe.” which is why one of my side projects is an attempt to place an entire degree’s worth of work into a 12-month span as a practice exercise for discussing the second and third year curriculum review that we’re holding later on this year.

Our ‘standard’ estimate for any normal degree program is that a student is expected to have a per-semester load of four courses (at 3 units a course, long story) and each of these courses will require 156 hours from start to finish. (This is based on 10 hours per week, including contact and non-contact, and roughly 36 hours for revision towards examination or the completion of other projects.) Based on this estimate, and setting up an upper barrier of 40 hours/week, for all of the good research-based reasons that I’ve discussed previously, there is no way that I can just pick up the existing courses and drop them into a year. A three-year program has six semesters, with four courses per semester, which gives an hour burden of 24*156 = 3,744. At 40 hours per week, we’d need 93.6 weeks (let’s call that 94), or 1.8 years.

But, hang on, we already have courses that are 6-unit and span two semesters – in fact, we have enormous projects for degree programs like Honours that are worth the equivalent of four courses. Interestingly, rather than having an exam every semester, these have a set of summative and formative assignments embedded to allow the provision of feedback and the demonstration of knowledge and skill acquisition – does this remove the need to have 36 hours for exam study for each semester if we build the assignments correctly?

Let’s assume that it does. Now we have a terminal set of examinations at the end of each year, instead of every semester. Now I have 12 courses at 120 hours each and 12 at 156 hours each. Now we’re down to 3,312 – which is only 1.6 years. Dang. Still not there. But it’s ok, I can see all of you who have just asked “Well, why are you so keen on using examinations if you’re happy with summative assignments testing concepts as you go and then building in the expectation of this knowledge in later modules?” Let’s drop the exam requirement even further to a final set of professional level assessment criteria, carried out at the end of the degree to test high-level concepts and advanced skills. Now, of the 24 courses that a student sits, almost all assessment work has moved into continuous assessment mode, rich in feedback, with summative checkpoints and a final set of examinations as part of the four capstone courses at the end. This gives us 3,024 hours – about 1.45 years.

But this is also ignoring that the first week of many of these courses is required revision after some 6-18 weeks of inactivity as the students go away to summer break or home for various holidays. Let’s assume even further that, with the exception of the first four courses that they do, that we build this continuously so that skills and knowledge are reinforced as micro slides, scattered throughout the work, supported with recordings, podcasts, notes, guides and quick revision exercises in the assessment framework. Now I can slice maybe 5 hours off 20 of the courses (the last 20) – cutting me down by another 100 hours and that’s half a month saved, down to 1.4 years.

Of course, I’m ignoring a lot of issues here. I’m ignoring the time it takes someone to digest information but, having raised that, can you tell me exactly how long it takes a student to learn a new concept? This is a trick question as the answer generally depends upon the question “how are you teaching them?” We know that lectures are one of the worst ways to transfer information, with A/V displays, lectures and listening all having a retention rate less than 40%. If you’re not retaining, your chances of learning something are extremely low. At the same time, somewhere between 30-50% of the time that we’re allocating to those courses we already teach are spent in traditional lectures – at time of writing. We can improve retention (of both knowledge and students) when we use group work (50% and higher for knowledge) or get the students to practice (75%) or, even better, instruct someone else (up to 90%). If we can restructure the ’empty’ or ‘low transfer’ times into other activities that foster collaboration or constructive student pedagogy with a role transfer that allows students to instruct each other, then we can potentially greatly improve our usage of time.

If we use this notion and slice, say, 20 hours from each course because we can get rid of that many contact hours that we were wasting and get the same, if not better, results, we’re down to 2,444 hours, about 1.18 years. And I haven’t even started looking at the notion of concept alignment, where similar concepts are taught across two different concepts and could be put in one place, taught once, consistently and then built upon for the rest of the course. Suddenly, with the same concepts and a potentially improved educational design – we’re looking the 1-year degree in the face.

Now, there will be people who will say “Well, how does the student mature in this time? That’s only one year!” to which my response is “Well, how are you training them for maturity? Where are the developing exercises? The formative assessment based on careful scaffolding in societal development and intellectual advancement?” If the appeal of the three-year degree is that people will be 19-20 when they graduate, and this is seen as a good thing, then we solve this problem for the 1-year degree by waiting two years before they start!

Having said all of this, and believing that a high quality 1-year degree is possible, let me conclude by saying that I think that it is a terrible idea! University is more than a sequence of assessments and examinations, it is a culture, a place for intellectual exploration and the formation of bonds with like-minded friends. It is not a cram school to turn out a slightly shell-shocked engineer who has worked solidly, and without respite, for 52 weeks. However, my aim was never actually to run a course in a year, it was to see how I could restructure a course to be able to more easily modularise it, to break me out of the mental tyranny of a three- or four-year mandate and to focus on learning outcomes, educational design and sound pedagogy. The reason that I am working on this is so that I can produce a sound course structure with which students can engage, regardless of whether they are full-time or not, clearly outlining dependency and requirements. Yes, if we break this up into part-time, we need to add revision modules back in – but if we teach it intensively (or on-line) then those aren’t required. This is a way to give students choice and the freedom to come in at any age, with whatever time they have, but without sacrificing the quality of the underlying program. This is a bootstrap program for a developing nation, a quick entry point for people who had to go to work – this is making up for decades of declining enrolments in key areas.

This is going on a war footing against the forces of ignorance.

There are many successful “Open” universities that use similar approaches but I wanted to go through the exercise myself, to allow me the greatest level of intellectual freedom while looking at our curriculum review. Now, I feel that I can focus on Knowledge Areas for my specifics and on the program as a whole, freed of the binding assumption that there is an inevitable three-year grind ahead for any student. Perhaps one of the greatest benefits for me is the thought that, for students who can come to us for three years, I can put much, much more into the course if they have the time – and these things, of interest, regarding beauty, of intellectual pursuits, can replace some of the things that we’ve lost over the years in the last two decades of change in University.


Brief but good news

A happy surprise in my mailbox today, but first the background. We’ve been teaching Puzzle Based Learning at Adelaide for several years now, based on Professor Zbigniew Michalewicz’s concept for a course that encouraged problem solving in a domain-free environment. (You can read more details about it by searching for Puzzle Based Learning with the surnames Falkner, Michalewicz and Sooriamurthi – we’ve had work published on this in IEEE Computer and as a workshop at SIGCSE, among several others.) Zbyszek (Adelaide), Raja (Sooriamurthi, a Teaching Professor at CMU) and I teamed up with Professor Ed Meyer (Physics at Baldwin-Wallace) to put together a textbook proposal to help people teach this information.

Great news – our proposal has been accepted by an excellent publishing house who appear to be genuinely excited about the book! As this is my first book, I’m very excited and pleased – but it’s a great reflection on the strength of the team and our composite skills and background, especially with the inter-disciplinary aspects. I’ve seen a lot of exciting work come out of Baldwin-Wallace and, while this is my first time working with Ed, I’m really looking forward to it. (Zbyszek, Raja and I have worked together a lot but I’m still excited to be working with them again!)

Good news after a rather difficult week.


Reflecting on rewards – is Time Banking a reward or a technique?

The Reward As It Is Often Implemented
(In Advance and Starting With a B)

Enough advocacy for a while, time to think about research again! Given that I’ve just finished Alfie Kohn’s Punished by Rewards, and more on that later, I’ve been looking very carefully at everything I do with students to work out exactly what I am trying to do. One of Kohn’s theses is that we tend to manipulate people towards compliance through extrinsic tools such as incentives and rewards, rather than provide an environment in which their intrinsic motivational aspects dominate and they are driven to work through their own interest and requirements. Under Kohn’s approach, a gold star for sitting quietly achieve little except to say that sitting quietly must be so bad that you need to be bribed, and developing a taste for gold stars in the student. If someone isn’t sitting quietly, is it because they haven’t managed sitting quietly (the least rewarding unlockable achievement in any game) or that they are disengaged, bored or failing to understand why they are there? Is it, worse, because they are trying to ask questions about work that they don’t understand or because they are so keen to discuss it that they want to talk? Kohn wants to know WHY people are or aren’t doing things rather than just to stop or start people doing things through threats and bribery.

Where, in this context, does time banking fit? For those who haven’t read me on this before, time banking is described in a few posts I’ve made, with this as one of the better ones to read. In summary, students who hand up work early (and meet a defined standard) get hours in the bank that they can spend at a later date to give themselves a deadline extension – and there are a lot of tuneable parameters around this, but that’s the core. I already have a lot of data that verifies that roughly a third of students hand in on the last day and 15-18% hand up late. However, the 14th highest hand-in hour is the one immediately after the deadline. There’s an obvious problem where people aren’t giving themselves enough time to do the work but “near-missing” by one hour is a really silly way to lose marks. (We won’t talk about the pedagogical legitimacy of reducing marks for late work at the moment, that’s a related post I hope to write soon. Let’s assume that our learning design requires that work be submitted at a certain time to reinforce knowledge and call that the deadline – the loss, as either marks or knowledge reinforcement, is something that we want to avoid.)

But, by providing a “reward” for handing up early, am I trying to bribe my students into behaviour that I want to see? I think that the answer is “no”, for reasons that I’ll go into.

Firstly, the fundamental concept of time banking is that students have a reason to look at their assignment submission timetable as a whole and hand something up early because they can then gain more flexibility later on. Under current schemes, unless you provide bonus points, there is no reason for anyone to hand up more than one second early – assuming synchronised clocks. (I object to bonus points for early hand-in for two reasons: it is effectively a means to reward the able or those with more spare time, and because it starts to focus people on handing up early rather than the assignment itself.) This, in turn, appears to lead to a passive, last minute thinking pattern and we can see the results of that in our collected assignment data – lots and lots of near-miss late hand-ins. Our motivation is to focus the students on the knowledge in the course by making them engage with the course as a whole and empowering themselves into managing their time rather than adhering to our deadlines. We’re not trying to control the students, we’re trying to move them towards self-regulation where they control themselves.

Secondly, the same amount of work has to be done. There is no ‘reduced workload’ for handing in early, there is only ‘increased flexibility’. Nobody gets anything extra under this scheme that will reinforce any messages of work as something to be avoided. The only way to get time in the bank is to do the assignments – it is completely linked to the achievement that is the core of the course, rather than taking focus elsewhere.

Thirdly, a student can choose not to use it. Under almost every version of the scheme I’ve sketched out, every student gets 6 hours up their sleeve at the start of semester. If they want to just burn that for six late hand-ins that are under an hour late, I can live with that. It will also be very telling if they then turn out to be two hours late because, thinking about it, that’s a very interesting mental model that they’ve created.

But how is it going to affect the student? That’s a really good question. I think that, the way that it’s constructed, it provides a framework for students to work with, one that ties in with intrinsic motivation, rather than a framework that is imposed on students – in fact, moving away from the rigidly fixed deadlines that (from our data) don’t even to be training people that well anyway is a reduction in manipulate external control.

Will it work? Oh, now, there’s a question. After about a year of thought and discussion, we’re writing it all up at the moment for peer review on the foundations, the literature comparison, the existing evidence and the future plans. I’ll be very interested to see both the final paper and the first responses from our colleagues!