Oh No, Major Spam Offensive!

Posted: July 21, 2012 | Author: | Filed under: Education, Opinion | Tags: | Leave a comment

My WordPress comment feed just exploded with spam and I had to make a choice between deleting a bucketload of spam by inspecting them individually (and not writing tomorrow’s post) or hitting the ‘throw it all away’ button.

If you sent me a comment in the last 24 hours and you are not a bot, please try again!

If you are a bot, my, but you’re getting clever. Perhaps… too… clever.

Relationship Management: Authenticity

Posted: July 21, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , , , , | 5 Comments

(Edit note: I tried to use a formatting mechanism that would make the e-mail examples stand out but in broke things for people with different browsers and for me on mobile browsers. I’ve switched it back to normal text and indented for clarity.)

I belong to the Qantas Frequent Flyer program and have a reasonable amount of status. The last time I hit ‘Gold’, they sent me a letter telling me about all of the perks if I then went to ‘Platinum’. This struck me as curious because, by doing so, they immediately reduced the reward of ‘going Gold’ (because it was now second best) and completely failed to show me that they had looked at my flying habits. To go to ‘Platinum’, I would have had to take all of the flights I just took – AGAIN. So, now, thanks to an ill-thought out letter I’m aware of two things: firstly, that Gold is for dummies and that the cool kids are Platinum, and, secondly, that the airline I’ve been flying with since the mid-90s doesn’t regard me as serious enough to track. It makes you question the relationship.

Now it’s not as if I’d actually expended any effort to go ‘Gold’, I’d just sat on a lot of Qantas planes, watched a lot of Futurama and Big Bang Theory, and accumulated points. What Qantas sent me was a message that basically said “Hey, just fly twice as often as you and, because you fly discount economy and we don’t give you that much for it, that means we want you to spend about 3 months of the year in the air. In Economy long haul.” That’s a bit irritating because, as someone who works with computers, it’s pretty easy to look at things like accrual rate, current time of the year and my flying pattern and realise that you were sending me the aviation equivalent of “Hey, you made your mortgage payment, want to buy Paris?”

There’s a lot of lip service given to the idea of relationship management and, while it’s easy to talk about, it’s hard to do. There’s a great deal of difference between sending students an e-mail if they’re not attending and trying to actually make a connection with the student. One of these can be done with a message like this:

From: Nick Falkner
To: Nick Falkner
BCC: list of students but put in to the mail message in a way that doesn’t show up.

Hey, I noticed that you haven’t been showing up in class for a while and that you also haven’t handed up a number of assignments. If you’d like to get in touch, please see me after class or send me an e-mail to organise a time.

Regards, Nick.

Now, this is, to me, disingenuous, because while it may all be true, it looks like it’s a personal message when it’s really a form letter. Hand on heart, yes, I’ve done this but, on reflection, it’s not really good enough. Yes, any attempt to get in touch with a student is better than nothing, but this has no personalisation to it. (Yes, large classes can be hard to personalise. We ran a course for 360 engineers and we had weekly assignments with a marking load of 36 hours. We had to use team marking, with me as quality control and arbiter. Because each student got the same marker each time, we managed to maintain a relationship through personalised feedback and consistency that would have been hard to manage with only one person – but, obviously, students in different blocks could have different experiences and we did have to swap in/out more than one marker.)

I spend a lot of time establishing relationships with my students but that means that I then have to spend a lot of time maintaining the relationships with my students. Even in large classes, if I’ve spoken to someone once, they expect me to remember their names! (And I certainly try to – I don’t always succeed but I’ve got better at it with practice.)

Even those students I haven’t yet managed to develop a relationship with can benefit from my attempts to try. So this is probably much closer to what I try to send. (My explanatory notes on this are also attached after two dashes — and in italics.)

From: Nick Falkner
To: Student Name — E-mail is to the student, not an anonymous list
CC: Any other lecturers in the course — This is so that the student knows that all lecturers are getting this info.

Dear Firstname, — This can be hard to know, even when you can see the full name, due to cultural issues. If you make a fair stab, most people help you out.
I was looking at the course “Underwater Knitting in Perl” and you haven’t submitted any work for assignments 2 and 3. I was wondering if you there was something that you wanted to talk about? If you have medical or compassionate extension requests for this time, then you do need to let me know, as we need to work out an alternative submission schedule if that’s appropriate. As a reminder, you do need to obtain at least 40% of the available marks in the assignment work component to pass but you can easily get back on track if you start doing the work again now.
— It’s not too late but it can be too late! You may need help! Can I help you?
If you’d like to talk to me in person, I have an office drop-in time from 2-4pm on Friday, and you can find me in office 9.99, Building 4, Third Circle, or you can call me on xxxxx if that’s easier. Obviously, e-mail is always great as that gets me wherever I am – but I don’t promise to reply immediately to e-mail sent at midnight! — How to get me! I also reserve the right to be inject humour randomly. 🙂
Are you available on Friday at 2pm? If so, please let me know.
— Easy question to answer. Last thing the student reads. Need to keep it short so it can be read quickly and easily. This may, actually, be slightly too long.

This isn’t perfect, obviously, and I’m sure I’ve broken any number of good rules by doing this but the most important thing is that the tone is very different. I’ve thought about this student and my concern appears more authentic because it is more authentic. Of course, it took me much longer to write but the chances of having a positive response are far greater. It’s also based on my knowledge of the student which, right now, is a little limited but at least I’ve dug up as much as I can. I’ve reminded them of the mechanisms that are in place to help, as an introductory step, without saying that there’s anything wrong with them and I’ve given them a reason to respond (you may put yourself at risk but it’s not too late) and a direct question (can you see me on Friday) to respond to.

 Unsurprisingly, the students who respond and stay in touch are usually the ones that I have the best relationship with – but the first e-mail and your demonstration of knowledge of the student, as well as the personalisation of the message, makes a difference. Today, I received an e-mail from someone who came to me with problems but, with some good e-mail and meetings, we got him through to a solid B – when he was on his way to an F. Here’s what he wrote:
[…] thanks for your help during the semester, without it I wouldn’t have been able to pass [the course]. I really appreciate it. I was actually a bit surprised to even manage a [B], so again cheers.
And that, I think, is all I really need to say on this.

The Extrinsic Reward: As Seen in the Wild.

Posted: July 20, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , | Leave a comment

“Why should I do it? What’s in it for me?”

How many times have you heard, said or thought the above sentiment, in one form or another? I go to a lot of meetings so I get to hear this one a lot. Reanalysing my interactions with people over the past 12 months or so, it has become apparent how many people are clearly focused on the payoff, and this is usually not related to their intrinsic reward mechanisms.

We get it from students when they ask “Will this be on the test?” (Should I study this? What’s in it for me?) We get it from our colleagues when they look at a new suggestion and say “Well, no-one’s going to do that.” (Which usually means “I wouldn’t do it. What’s in it for me?”) We get it from ourselves when we don’t do something because something else becomes more important – and this is very interesting as it often gives an indicator of where you sit on the work/life balance scale. Where I work, there are a large number of occasions where the rewards mechanisms used can result in actions and thinking patterns that, as an observer, I find both interesting and disturbing.

Let me give you some background on how research funding works in Australia (very brief). You have a research idea or are inside a group that has some good research ideas. You do research. You discover something. You write it up and get it published in conferences and journals. Repeat this step until you have enough publications to have a credible track record. You can now apply for funding from various bodies, so you spend 3-4 weeks writing a grant and you write up your great grant idea, write it up really well, attach your track record evidence as part of your CV, and then wait. In my discipline, ICT, our success rate is very low, and very few of the people who apply for Australian Research Council Discovery Grants get their grants. Now this is, of course, not a lottery – this is a game of skill! Your grant is rated by other people, you get some feedback, you can respond to this feedback (the rejoinder), and the ratings that you originally received, plus your rejoinder, go forward to a larger panel. Regrettably, there is not much money to go around (most grants are only funded at the 50% level of the 22% of grants that get through across the board), so an initial poor rating means that your grant is (effectively) dead.

This makes grants scarce and intrinsically competitive, as well as artificially inflated in their perceived value. Receiving a grant will also get you public congratulations, the money and gear (obviously) and an invitation to the best Christmas cocktail party in the University – the Winner’s Circle, in effect. The same is true if you bring in a heap of research cash of any other kind – public praise, money and networking opportunities.

Which, if you think about it, is rather curious because you have just been given a wodge (technical term) of cash that you can use to hire staff and buy gear, travel to conferences, and basically improve your chances of getting another grant – but you then get additional extrinsic rewards, including the chance to meet the other people who have risen to this level. This is, effectively, a double reward and I suppose I wouldn’t have much of a problem with it, except that we start to run into those issues of extrinsic motivation again which risks robbing people of their inclination to do research once those extrinsic rewards dry up. I note that we do have a scheme to improve the grant chances of people who just missed out on getting Australian Research Council (ARC) funding but it is literally for those people who just missed out.

Not getting a grant can be a very negative result, because the absence of success is also often accompanied by feedback that will force you to question the value of your performance to date, rather than just the work that has been submitted.

When an early career researcher looks at the ARC application process and thinks “What’s in it for me?” – the answer is far more likely to be “an opportunity to receive feedback of variable quality for the investment of several weeks of your life, from people with whom you are actively competing” rather than an actual grant. So this is obviously a point where mentoring, support and (yes) seed funding to be able to improve become very important – as it provides an ability to develop skill, confidence and (hopefully) the quality of the work, leading to success in the future. The core here, however, is not to bribe the person into improving, it’s to develop the person in order that they improve. Regrettably, a scheme that is (effectively) rewarding the rewarded does not have a built-in “and lifting up those who aren’t there” component. In fact, taking on a less experienced researcher is far more likely to hinder a more capable applicant’s chances. When a senior researcher looks at assisting a more junior researcher, under the current system, “What’s in it for me?” is mostly “Reduced chance of success.” Given that this may also cut you out of the Winner’s Circle, as funds dry up, as you are no longer successful, as it then gets harder to do the research and hence get grants, combined with the fact that you can only apply for these once a year… it’s a positive disincentive to foster emerging talent, unless that talent is so talented that it probably doesn’t need that much help!

So the extrinsic manipulation here has a built-in feedback loop and is, regrettably, prone to splitting people into two groups (successful and not) very early on, at the risk of those groups staying separated for some time to come.

If the large body of work in the area is to be believed, most people don’t plan with the long term outcomes in mind (hence, being told that if you work hard you might get a grant in five years is unlikely to change anyone’s behaviour) and on top of that, as Kohn posits, praising a successful person is more likely to cause envy and division than any real improvement. How does someone else being praised tell you how to improve from your current position?

So what does all of this hot air mean for my students?

I have just finished removing all ‘attendance-based’ incentive schemes from my courses – there are no marks being given just for showing up in any form, marks are only achieved when you demonstrate that you have acquired knowledge. Achievement will not generate any additional reward – the achievement will be the reward. Feedback is crucial but, and this will be challenging, everything I say or do must provide the students with a way to improve, without resorting to the more vague areas of general praise. I will be interested to see if this appears to have any (anecdotal) effect upon the number of times someone asks “What’s in it for me?”

Intervention and Risk: An Anonymised Anecdote

Posted: July 19, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , | Leave a comment

Yesterday morning, we found some students sleeping in one of our computing labs. This isn’t that uncommon, especially during the crunch times, but it is uncommon to see people disrobed and obviously moved in, with food, clothes and the like. The initial reactions are almost always “Argh, what are you doing in here?” and “Grr, have you been getting in the way of other students.” However, and I can’t go into too much detail, as the story unfurled, with the intervention of some excellent staff members who managed to get the students talking, what appeared to be students taking advantage of our resources quickly turned out to be a situation where one student in extremis was being watched and cared for by another student – while both students were dealing with other, far more serious, problems.

To put it simply, one student had almost run out of hope and places to be. When you think about it, you’re not going anywhere good when you end up hiding in the corner of a lab that’s going through software rebuild and, hence, has no-one in it. The initial problem that we had was that, for mainly cultural reasons, the students had a great deal of difficulty talking to the first people to contact them – because we were lecturers and there is a great deal of potential embarrassment for certain people in admitting to problems in front of us. Fortunately, many heads knocked together to look at the problem, someone managed to start the students talking, we got more information and, as of this morning, a number of key problems have been solved. The major issue (stress regarding study) has been dealt with and the intervention to address other problems continues.

Reflecting upon this situation, I was reminded again of the burden that is placed upon the relationship between student and staff member when there is a cultural gap, especially one involving academic staff. I tried to talk to the students but, having been set up into fixed roles (in their heads), we couldn’t communicate. It was only once someone outside of the academic hierarchy got involved that information started to flow. Yes, there were linguistic issues but, ultimately, it didn’t come down to language, it came down to willingness to talk and these students didn’t want to open up. After they were reached, then the vast array of helpful resources that we do have were suddenly available to them.

As was noted at HERDSA recently, students don’t look at the ‘where to go for help’ slides early on in a course because they don’t need help. If students do need help, but can’t ask for it or don’t know where to go, then all of our helpful and assistive systems just won’t be able to help. But, of course, expecting students to know when they need help does give us a convenient ‘out’. Given that we can see their marks, and to a large extent their academic performance in courses that we administer, we should be able to see students who are heading towards crisis points. (We do look at this in our Faculty but more on that later.)

My own research, to be presented at ICER in September, talks about the amount of information that appears to be contained in the first submission that a student makes. But let’s say that all I can see is a semester of Fail grades – given that performance like that wouldn’t have got them into my course, I’m looking at a problem. Now, we can and we do redirect students to our (very good) Transitions and Advisory Service but this is a manual step. I’ve been looking at automated solutions to this for some time, and I’m looking forward to talking to people in more detail about AWE (the Wellness Engine) at University of New England, because I should not have to use myself as a processing element in order to achieve something that can be done better by a computer.

A colleague and friend of mine was describing middleware to some people at the University. If you don’t know what it means, middleware is software that connects two or more other systems together. Rather than writing one big piece that does everything, or two pieces that fit together like a jigsaw, middleware allows you to bring together lots of different systems that weren’t necessarily designed to work with each other. Probably the example that you’ve seen, and not realised, is using a database through a web-page. The underlying data (like Amazon’s store) is one system. Your web browser is another. Middleware allows you to exchange data with the data store and buy books. Middleware sounds great, right? It is – but here comes the catch.

Dave’s killer question on this is “Why are we using our staff as middleware?”

He’s right, of course. We take data from our marking of assignments, put into another system (by changing format and restructuring it), then we put that into another system (with manual intervention and checking) and this is then finally made available to students. Now if I want to see how the students are doing, I need to remember to manually request that a search be made, showing me all students who have failed anything – and then give me their GPA for this semester. I note that we already do this at the Faculty level using a mechanism called the Unsatisfactory Academic Progress process, which has identified a lot of at-risk students and helped a lot of people back, but how is it done? People acting as middleware.

What I want is a system that alerts me to problems automatically. If I have to search, it takes time and (worse) it becomes a task to be prioritised because there many not always be problems. If I am contacted when there is a problem, the task is automatically high priority. That requires a good set of middleware that spans all of a University’s systems and can bring that data together, then get in touch with the right people when there’s a problem. We’re actually not that far away from it – the systems are all there, we just need to streamline some processes. Fewer people acting as middleware means more people doing the things that we actually pay them for, especially when it’s academics!

There are lots of things that can get in the way of a good working relationship between educator and student. We don’t have to be friends, but we do have to be willing and able to talk to each other. Taking that further, it would be nice if the systems all talked to each other as well, including yelling at us when a student hits a mark where we might be able to intervene and do something useful, sooner.

A Design Challenge, a Grand Design Challenge, if you will.

Posted: July 18, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , | 1 Comment

Question: What is one semester long, designed as a course for students who perform very well academically, has no prerequisites and can be taken by students with no programming exposure and by students with a great deal of programming experience?

Answer: I don’t know but I’m teaching it on Monday.

While I talk about students who perform well academically, this is for the first instance of this course. My goal is that any student can take this course, in some form, in the future.

The new course in our School, Grand Challenges in Computer Science, is part of our new degree structure, the Bachelor of Computer Science (Advanced). This adds  lot more project work and advanced concepts, without disrupting the usual (and already excellent) development structure of the degree. One of the challenges of dealing with higher-performing students is keeping them in a sufficiently large and vibrant peer group while also addressing the minor problem that they’re moving at a different pace to many people that they are friends with. Our solution has been to add additional courses that sit outside of the main progression but still provide interesting material for these students, as well as encouraging them to take a more active role in the student and general community. They can spend time with their friends, carry on with their degrees and graduate at the same time, but also exercise themselves to greater depth and into areas that we often don’t have time to deal with.

In case you’re wondering, I know that some of my students read this blog and I’m completely comfortable talking about the new course in this manner because (a) they know that I’m joking about the “I don’t know” from the Answer above and (b) I have no secrets regarding this course. There are some serious challenges facing us as a species. We are now in a position where certain technologies and approaches may be able to help us with this. One of these is the notion of producing an educational community that can work together to solve grand challenges and these students are very much a potential part of this new community.

The biggest challenge for me is that I have such a wide range of students. I have students who potentially have no programming background and students who have been coding for four years. I have students who are very familiar with the School’s practices and University, and people whose first day is Monday. Of course, my solution to this is to attack it with a good design. But, of course, before a design, we have to know the problem that we’re trying to solve.

The core elements of this course are the six grand challenges as outlined but he NSF, research methods that will support data analysis, the visualisation of large data sources as a grand challenge and community participation to foster grand challenge communities. I don’t believe that a traditional design of lecturing is going to support this very well, especially as the two characteristics that I most want to develop in the students are creativity and critical thinking. I really want all of my students to be able to think their way around, over or through an obstacle and I think that this course is going to be an excellent place to be able to concentrate on this.

I’ve started by looking at my learning outcomes for this course – what do I expect my students to know by the end of this course? Well, I expect them to be able to tell me what the grand challenges are, describe them, and then provide examples of each one. I expect them to be able to answer questions about key areas and, in the areas that we explore in depth, demonstrate this knowledge through the application of relevant skills, including the production of assignment materials to the best of their ability, given their previous experience. Of course, this means that every student may end up performing slightly differently, which immediately means that personalised assessment work (or banded assessment work) is going to be required but it also means that the materials I use will need to be able to support a surface reading, a more detailed reading and a deep reading, where students can work through the material at their own pace.

I don’t want the ‘senior’ students to dominate, so there’s going to have be some very serious scaffolding, and work from me, to support role fluidity and mutual respect, where the people leading discussion rotate to people supporting a point, or critiquing a point, or taking notes on the point, to make sure that everyone gets a say and that we don’t inhibit the creativity that I’m expecting to see in this course. I will be setting standards for projects that take into account the level of experience of each person, discussed and agreed with the student in advance, based on their prior performance and previous knowledge.

What delights me most about this course is that I will be able to encourage people to learn from each other. Because the major assessment items are all unique to a student, then sharing knowledge will not actually lead to plagiarism or copying. Students will be actively discouraged from doing work for each other but, in this case, I have no problem in students helping each other out – as long as the lion’s share of the work is done by the main student. (The wording of this is going to look a lot more formal but that’s a Uni requirement. To quote “The Castle”, “It’s about the vibe.”) Students will regularly present their work for critique and public discussion, with their response to that critique forming a part of their assessment.

I’m trying to start these students thinking about the problems that are out there, while at the same time giving them a set of bootstrapping tools that can set them on the path to investigation and (maybe) solution well ahead of the end of their degrees. This then feeds into their project work in second and third year. (And, I hope, for at least some of them, Honours and maybe PhD beyond.)

Writing this course has been a delight. I have never had so much excuse to buy books and read fascinating things about challenging issues and data visualisation. However, I think that it will be the student’s response to this that will give me something that I can then share with other people – their reactions and suggestions for improvement will put a seal of authenticity on this that I can then pack up, reorganise, and put out into the world as modules for general first year and high school outreach.

I’m very much looking forward to Monday!

Good Design: Building In Important Features From the Start

Posted: July 17, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , | Leave a comment

The game “Deus Ex” is widely regarded as one of the best computer games that has been made so far. It has won a very large number of “best game” awards and regularly shows up in the top 5 of lists of “amazing games”. Deus Ex was released in 2000, designed and developed by Ion Storm under Warren Spector and Harvey Smith and distributed by Eidos. (I mentioned it before in this post, briefly.) Here is the description of this game from Wikipedia:

Set in a dystopian world during the year 2052, the central plot follows rookie United Nations Anti-Terrorist Coalition agent JC Denton, as he sets out to combat terrorist forces, which have become increasingly prevalent in a world slipping ever further into chaos. As the plot unfolds, Denton becomes entangled in a deep and ancient conspiracy, encountering organizations such as Majestic 12, the Illuminati, and the Hong Kong Triads throughout his journey.

Deus Ex had a cyberpunk theme, a world of shadowy corporations and many corruptions of the human soul, ranging from a generally materialistic culture to body implants producing cyborg entities that no longer had much humanity. While looking a lot like a First-Person Shooter (you see through the character’s eyes and kill things), the game also had a great deal of stealth play (sneaking around trying very hard not to get noticed, shot or both). However, what sets DE apart from most other games it that the choice of how you solved most of the problems was pretty much left up to you. This was no accident. The fact that you could solve 99% of the problems in the game by using different forms of violence, many forms of stealth or a combination of these was down to the way that the game was designed.

When I was at Game Masters at ACMI, Melbourne, over the weekend, I was able to read the front page of a document entitled “Just What IS Deus Ex” by Warren Spector. Now, unfortunately, they had a “no photographs” rule so I don’t have a copy of it (and, for what it’s worth, I also interpreted that to mean “no tiresome hand transcription onto the iPhone in order to make a replica” ) but one of the most obvious and important design features was that they wanted to be able to support player exploration: players’ actions had to have consequences and players needed to be able to make their plans, without feeling constrained by the world. (Fortunately, while not being the actual document, there is an article here where Warren talks about most of the important things. If you’re interested in design, have a look at it after you’ve finished this.) Because of this, a number of the items in the game can be used in a number of quite strange ways and, while it appears that this is a bug, suddenly you’ll run across an element of the game that makes you realise that the game designers knew that this was possible.

Do not climb if red lights active!

For example, in the Triad-run Hong Kong of 2052, there is a very tall tower on one edge of the explorable area. There are grenades (LAMs)in the game that adhere ‘magnetically’ to walls and then explode if armed and someone enters their proximity. However, it is possible to use these grenades to climb up walls, assuming you don’t arm them of course, by sticking them to walls, getting close enough to hop up, placing another grenade above you and then doing the same thing. With patience, you can climb quite high. Sounds like a bug, right? Yeah, well, that’s what I thought until I climbed to the top of the tower in Hong Kong and found a guy, one of the Non-Player Characters, standing on top.

This was a surprise but it shouldn’t have been. I’d already realised that there was always more than one way to do things and, because the game was designed to make it is as easy as possible for me to try many paths to achieve success, the writer had put in early hints designed to discourage a ‘blow everything up’ approach. The skill system makes it relatively easy for you to make your life a lot easier by working with what is already in the environment rather than trying to do it all yourself.

In terms of the grenades, rather than just being pictures on a wall, they became real world objects when placed and were as solid as any other element. This allowed them to be climbed and the designers/programmers recognised this by putting a guy on top of a tower that you had no other way to get to (without invoking cheats). The objects in Deus Ex were designed to be as generally usable as possible. The sword could open crates as well (Ok, well much better) than a crowbar could and reduced the need to carry two things. Many weapons came with multiple ammunition types, allowing you to customise your load out to the kind of game you wanted to play. Other nice features included the fact that there very few situations of ‘spontaneous creation’, where monsters appeared at some point in a scripted scene, which would have enforced a certain approach. If you were crawling in somewhere from completely the wrong side, everything would be there and ready, rather than all spontaneously reappearing when you happened to approach from the ‘triggering’ side.

In short, it felt like a real world. (With the usual caveat regarding it being a real world where you are a killer cyborg in 2052.)

The big advantage of this is that you feel a great deal of freedom in your planning and implementation and, combined with the fact that the game reacts and changes to the decisions that you make, this makes the endings of the game feel very personal – when you finally choose between the three possible endings, you do so feeling like the game is actually going along with the persona that you have set up. This increases the level of engagement, achievement and enjoyment.

One of Mark Guzdial’s recent posts talked about the importance of good design when it comes to constructing instructional materials and I couldn’t agree more. Good design at the start, with a clear idea of what you’re trying to achieve, allows you to build a consistent experience that will allow you and your students to achieve your objectives. Deus Ex is, in my opinion, considered one of the best games of the 21st century because it started from a simple and clear design document that was set out to maximise the degree of influence that the player could feel in the game – everyone who plays Deus Ex takes their own path through it, has their own experience and gets something slightly different out of it.

I’m not saying it’s that easy for educational design as a global issue, but it is a very good reminder of why we should be doing good design at the very beginning of our courses!

Puppet on a String: A Summary of My Corruption by Extrinsic Rewards

Posted: July 16, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , | 3 Comments

I recently posted that I was thinking about my own contributions and asking what, if anything, would denote something that could be recognised as my mastery of my discipline. On thinking about this, I realised that, once again, I was asking someone else to value my work. For those of you who are educational specialists, rather than a discipline researcher who is on his way to becoming an educational researcher within the discipline, this is probably somewhat amusing, given I keep talking about the need to reduce extrinsic motivation in my students.

I have changed career several times and, if you look at why I’ve done this, a pattern quickly emerges. I tend to leave at the point where I have become competent enough that other people start to tell me that what I am doing is useful, valuable and start trying to reward me. Yet, I go into jobs seeking that kind of recognition and reward. I am corrupted in my intent, by the rewards, and then my intrinsic reward mechanisms become compromised and, after becoming deeply unhappy, I leave.

I realised, over the weekend, that I was becoming so pre-occupied with external approval that it was making me extremely vulnerable to criticism and it was corrupting me in trying to do something that is, whether I like it or not, very important and that I also happen to be good at.

Right now, I am in the middle of trying to work out how to divorce myself from the external rewards that I, irritatingly, crave and that, ultimately, then reduce the joy I take in doing things for my own reasons. It’s not surprising that the tasks that I enjoy the most at the moment are the big challenges, the ones where I’m working several levels above my pay grade or the usual expectations of someone of my level. I’m doing these things because they’re important and, because I’m doing it ‘out of cycle’ so to speak, I can’t be externally rewarded for them – I can just do a good job.

It’s in this same mode of thinking that I’ve decided not to spend any time applying for any local teaching and excellence awards. (I was about to comment on my potential eligibility but this is just another quest for a pat on the head – so I’ve deleted it.) I am either doing my job in the way that I should, and the expectations should be of a satisfactory performance that provides students with an excellent experience, or I should receive guidance, counselling and remedial assistance from my employer. Ultimately, if I don’t meet the standards then I should probably be fired. But if I’m doing well, then that is my job and I don’t need a piece of paper or a cheque to make things better. In fact, that money and time (in deciding upon the awards or writing the applications) should be directed to people who need the improvement, not people who are excelling. I have a meeting with my boss on Friday week and he will tell me whether I’m meeting standard or not.

Now there is a great deal of difference between writing a long application for an award (which is probably not the best investment of time and is seeking extrinsic recognition) and being sent on a course that might be useful because you’ve demonstrated an ability to do something (providing you with useful skills and the ability to develop further). As a general principle, skill development is going to be more useful than a pat on a head. Skill development also works for everyone, it’s just that the courses you use for development vary from person to person.

But this is, of course, completely at odds with the extensive systems of measurement that are now being placed on academics. We are (with widely varying levels of accuracy) measured extensively in terms of learning and teaching, research and administration. By not applying for these awards, I may be significantly altering my possibility of later promotion and opportunity. And, yet, I have to ask myself if I really need to be promoted? What does it mean? I’ve already discovered that people are happy to let you do a wide range of jobs without the requisite ‘academic level’ if you can demonstrate enough aptitude. Sure, it would mean I’d never be able to do certain jobs but, having a look at those jobs, I’m not sure that’s such a bad thing. 🙂

This is a strange time for me. I can now see the strings around me and how they’ve pulled me around for all of my life. Because I am such a strong believer in being as honest as possible with my students, it has forced me to be honest with myself, as I tear apart the framework I teach within to see how I can improve it and help my students to become self-regulated, intrinsically motivated and happy. Authenticity is the core for me and it is why I can teach with passion.

I was looking at Facebook recently and thinking about the “Like” button. I use it to mean “I am happy about this” or “I support you” but, rather than telling someone this, I hit the “Like” button. I’ve recently noticed that there are “Like” levels in WordPress and as I’ve hit, arbitrary, milestones I’ve received insincere automated badges.

Some of my readers (thank you, again) have been letting me know how they have been using the stuff from here and that has been really helpful for me. I realise that, in this community, “Like” generally means “I agree” or “Nicely written thoughts that ring true” but getting an actual account of how someone has used something that I said turned out to be really powerful. (Unsurprisingly, given how much Kohn I’m reading at the moment!)

So – where to from here? The first thing is to keep to my 40-45 hour working week. That has allowed me to get enough reflection time to get to this stage. I suspect the next is to keep plugging away at everything. This is most definitely not the time to throw everything in the air and meditate in a field. I’ve been trying to think about the advice that I would give to a student in a similar situation and I think I would tell them to keep doing everything and set some time aside over the next couple of weeks to identify the key issues, then start stripping away clutter until they were able to get a clear view of how they could achieve what was important to them. It will, at least, be a start.

Wrath of Kohn: Well, More Thoughts on “Punished by Rewards”

Posted: July 16, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , | 1 Comment

Yesterday, I was discussing my reading of Alfie Kohn’s “Punished by Rewards” and I was talking about a student focus to this but today I want to talk about the impact on staff. Let me start by asking you to undertake a quick task. Let’s say you are looking for a new job, what are the top ten things that you want to get from it? Write them down – don’t just think about them, please – and have them with you. I’ll put a picture of Kohn’s book here to stop you looking ahead. 🙂

It’s ok, I’ll wait. Written your list?

How far up the list was “Money”? Now, if you wrote money in the top three, I want you to imagine that this new job will pay you a fair wage for what you’re going to do and you won’t have any money troubles. (Bit of a reach, sometimes, I know but please give it a try.) With that in mind, look at your list again.

Does the word “excellent incentive scheme” or “great bonus package” figure anywhere on that list? If it does, is it in the top half or the bottom half? If Money wasn’t top three, where was it for you?

According to Kohn, very few people actually to make money the top of their list – it tends to be things like ‘type of work’, ‘interesting job’, ‘variety’, ‘challenge’ and stuff like that. So, if that’s the case, why do so many work incentive schemes revolve around giving us money or bonuses as a reward if, for the majority of the population, it’s not the thing that we want? Well, of course, it’s easy. Giving feedback or mentoring is much harder than a $50 gift card, a $2,000 bonus or 500 shares. What’s worse is, because it’s money, it has to be allocated in an artificial scarcity environment or it’s no longer a bonus, it’s an expectation. If you didn’t do this, then the company might go bankrupt.

What if, instead, when you did something really good, you received something that made it easier for you to do all of your work as a recognition of the fact that you’re working a lot? Of course, this would require your manager to have a really good idea of what you were doing and how to improve it, as well as your company being willing to buy you that backlit keyboard with the faux-mink armrest that will let you write reports without even a twinge of arm strain. Some of this, obviously, is part of minimum workplace standards but the idea is that you get something that reflects that your manager understands what you’ve done and is trying to help you to develop further. Carefully selected books, paid trips to useful training, opportunities to further display your skill – all of these are probably going to achieve more of the items on your 10-point list than money will. To quote Kohn, quoting Gruenberg (1980), “The Happy Worker: An Analysis of Educational and Occupational Differences in Determinants of Job Satisfaction”, American Journal of Sociology, 86, pp 267-8:

“Extrinsic rewards become an important determinant of overall job satisfaction only among workers for whom intrinsic rewards are relatively unavailable.”

There are, Kohn argues, many issues with incentive schemes as reward and one of these is the competitive environment that it fosters. I discussed this yesterday so I’ll move to one of the other, which is focusing on meeting the requirements for reward at the expense of quality and in a way that is as safe as possible. Let me give you an example that I recently encountered outside of work: Playing RockBand or SingStar (music games that score your performance). Watch me and my friends who actually sing playing a singing game: yes, we notice the score, but we don’t care about the score. We interpret, we mess around, we occasionally affect the voices of the Victorian-era female impersonator characters from Little Britain. Then watch other groups of people who are playing the game to make the highest score. They don’t interpret. They don’t harmonise spontaneously. In many cases, they barely even sing and focus on making the minimum tunefully accurate noise possible at exactly the right time, having learned the sequence, to achieve the highest score. The quality of the actual singing is non-existent, because this isn’t singing, it’s score-maximisation. Similarly, risk taking has been completely removed. (As an aside, I have excellent pitch and, to my ears, most people who try to maximise their score sound out of tune because they are within the tolerances that the game accepts, but by choosing not to actually sing, there is no fundamental thread of musicality that runs through their performance. I once saw a professional singer deliver a fantastic version of a song, only to be rated as mediocre by the system,)

On Saturday, my wife and I went to the Melbourne-based Australian Centre for the Moving Image (ACMI) to attend the Game Masters gaming exhibition. It was fantastic, big arcade section and tons of great stuff dedicated to gaming. (Including the design document for Deus Ex!) There were lots of games to play, including SingStar (Scored karaoke), RockBand (multi-instrument band playing with feedback and score) and some dancing games. Going past RockBand, Katrina pointed out how little fun the participants appeared to be having and, on looking at it, it was completely true. The three boys in there were messing around with pseudo-musical instruments but, rather than making a loud and joyful noise, they were furrowed of brow and focused on doing precisely the right things at the right times to get positive feedback and a higher score. Now, there are many innovations emerging in this space and it is now possible to explore more and actually take some risks for innovation, but from industry and from life experience, it’s pretty obvious that your perception of what you should be doing and where the reward is going to come from make a huge difference.

If your reward is coming from someone/something else, and they set a bar of some sort, you’re going to focus on reaching that bar. You’re going to minimise the threats to not reaching that bar by playing it safe, colouring inside the lines, trying to please the judge and then, if you don’t get that reward, you’re far more likely to stop carrying out that activity, even if you loved it before. And, boy, if you don’t get that reward, will you feel punished.

I’m not saying Kohn is 100% correct, because frankly I don’t know and I’m not a behaviourist, but a lot of this rings true from my own experience and his use of the studies included in his book, as well as the studies themselves, are very persuasive. I look forward to some discussion on these points!

A (Confusingly) Rewarding Read: Reading Kohn’s Punished By Rewards

Posted: July 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , | 4 Comments

I’ve been flying a lot and, as the electronic gadgets have to be off for a while, I’ve been carrying books to read. The one that held my interest on the flight from Adelaide to Darwin was Kohn’s “Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, As, Praise, and Other Bribes.” This is not a discipline text by any stretch of the imagination and, both as it is written for someone who isn’t very familiar with behaviourist terminology and is fairly critical of what it defines as behaviourism, it’s always important to be reserved in taking texts like this without reading around, looking for earlier critiques and being a little skeptical to start with. However, with that said, there’s a lot to think about in here.

Kohn’s fundamental thesis is that incentive schemes that are designed to manipulate or implicitly coerce us into ‘desirable’ behaviour tend to have the opposite effect. He presents a wide range of studies, unfortunately from a perspective that is fairly anti “behaviourism-of-the-60s-to-early-90s”, but his overall findings and reports are very interesting.

I’ve talked about how setting up extrinsic reward structures always runs the risk of making people optimise their behaviour for the reward, without regard to the desired behavioural change, and this, combined with a difficulty in quantifying what it is that people actually do in any way that assesses quality, tends to make people focus on doing the minimum possible to achieve the reward. In other words, pay people a bonus for producing the most letter boxes a day and, chances are, the person who gets the bonus may have made the worst letter boxes because they were focused on quantity and not quality. The other, and to me very counterintuitive, finding was that associating a reward with something that you previously found desirable can have the unintended consequence of making you value it less, either because if someone has to bribe you to do it, it can’t be that great, or because, like most people, you don’t like being manipulated and your intrinsic motivations are swamped by negative associations with the new extrinsic motivator.

Praise, itself, comes under the microscope as well, as it always carries the risk of making someone focus on pleasing you, rather than doing the job correctly. Praise is also often used as a manipulation tool to enforce compliance (Isn’t Johnny a good boy for sitting so quietly! Why, I think he’s the quietest boy in the room!) and this often leads to resentment, as well as the obvious divisions that occur when you praise one person/group and not other.

One of the most interesting points from this book, which we all know but often don’t consider, is that in an environment where praise is given, the absence of praise can be as negative as punishment because, by definition, if you’re not being praised then you’re not good enough. Rather than leading to some invisible-hand-led revolution into productivity and compliance, all too often this leads to resentment, defiant semi-compliance and disengagement.

Kohn discusses feedback, rather than praise, and focusing on the objective, rather than the subjective. This is not to say that he completely condemns praise, or for that matter reward-as-incentive, but he is strongly opposed to the widespread use that we see today. Yes, the interpretation of praise varies across people and age groups and it’s often  impossible to strip all emotional content from good feedback, but he suggests that being aware of it allows us to be less prone to over-praising, being seen as over-praising and focusing on the essential person (which is largely immutable) rather than a skill or practice that cam be enhanced with feedback. His specific suggestions, including his examples, are:

  1. Don’t praise the person, praise what they do: “That’s a good story” is a good way to discuss good work, rather than saying “You’re such a good writer”, which (especially with children) can be seen as insincere or patronising, especially if the child is aware of the divide between their works and those of others. Personal, rather than activity-driven, praise can lead to a loss of interest in the activity.
  2. Make praise as specific as possible. Focus on the act and call attention to the specific components that are innovative, as an example, or otherwise worthy of notice. “That’s a really nice story” says one thing but by saying “The ending is good where you leave the main character confused as to what happened to him” the student then can see what your standards are and contextualise your feedback.
  3. Avoid phony praise. If you’re genuinely thrilled by something, then (obviously) let people know, but Kohn advises against employing false praise when you catch someone in the act of doing something that you want them to do (behavioural reinforcement). Especially if you use that Glinda the Good Witch voice that most of us remember from our childhood, which is about as genuine and warming as a three dollar bill.
  4. Avoid praise that sets up a competition. Kohn suggests that saying “You’re the best in the class/school/department” in public has just divided the group into One Person and The Rest. Praising someone like this in public leads to competitive behaviour, which will inhibit cooperation, collaboration and all of the educational benefits that you could have obtained had you read your Vygotsky.

The sinister thing about item 4 is that praise handed out in this way gives a lot of power to the rating body. If you, as an individual educator, pick someone out as being the most well-behaved or quietest child, then you have told everyone in the room that they have to please YOU in order to get the nice praise reward. No one is actually reflecting on being quiet or well-beheaved, they’re thinking about what they have to do (most likely the minimum) in order to get that nice warm feeling from you.

Don’t believe me? When someone picks up a mistake in a presentation and suggests a correction, then if they are praised (maybe given a small prize, a jelly bean) Kohn’s thesis suggests that, if you have a subsequent error, more people will spot that mistake because they have switched their behaviour from learning from or listening to you to scanning the notes for mistakes in order to achieve extrinsic reward. I’ve certainly seen that behaviour in class and, worse, I can remember doing that in school when the ‘smart’ kids would get personally praised in class for picking up errors on the blackboard.

I’ll write some more on Kohn for tomorrow. I still have to finish the last of the book and then read through all of the critiques, I’ve read a couple, so I hope to write a later post on this from an opposing viewpoint.

Richard Hil on “Whackademia”: Conversations with Richard Fidler

Posted: July 14, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , | 2 Comments

The Australian national broadcaster’s Radio National station has a segment called Conversations with Richard Fidler, where the host has a roughly 30 minute chat with someone interesting. A friend (thanks, Cathy!) recently sent me a link to a conversation that Fidler had with Richard Hil, author of “Whackademia: An insider’s account of the troubled university.” She sent me the link of the 25th of June and it’s slightly telling that it took me until the 10th of July, when I was home sick and was looking for things to listen to, that I finally had the chance to devote 30 minutes to just sitting and listening. You can find the link to the podcast itself here, but I’m not sure if it will work outside of Australia. However, I have some thoughts on the podcast that will work just as well if you haven’t listened to it. The vast majority of the words here are my interpretation of Hil and Fidler. I will insert my own comments parenthetically.

Hil is fundamentally concerned with the change in Universities that he perceives as the change in the focus of education and making life less enjoyable and far less free for the academics. He feels that students have become, in the words of Laurie Taylor (from Schwartz’s “Paradox of Choice“), shoppers, in the sense that they are controlled consumers, shaped by marketing, branding and the illusion of choice. Students are becoming far less likely to spend time on campus, with up to a third surveyed reporting that they haven’t made even a single friend. Given the pressures of modern life, and fitting education in and around work and family, this is hardly surprising, especially when combined with the increasing on-line availability of courses.

One of the other elements in play is the strong vocational focus that drives subject and degree choice, students now being far less likely to take courses that are potentially enriching in the future, which often lifted universities up from the requisite lectures and tutorials and allowed students to be involved in the energy of education. It’s also very easy to scoff at “enrichment” courses, especially if it they are marketed in an empty or cynical fashion.

Some universities, such as Macquarie, have experimented with third year courses with titles like “Practical Wisdom” to cover general world, thinking and important issues – but at third year, we have to ask what we have been teaching up until then along similar lines? What are we doing to form the global citizen?

Hil identifies what he refers to as the rise of managers in University, increasing regulation, driving business-speak, business models and performance management drawn from, and more suited to, traditional private enterprise. (He makes a loose argument against the perceived subjectivity of performance reviews, but I didn’t feel it was very strong.) However, how did we get here? Was it the Dawkins Revolution? Hil thinks not.

Hil identifies an early essay by Milton Friedman about the role of University in society and the economy, which advocated a student loan system and a deregulation of the University sector to move it into a competitive business model. (My reading of the essay agrees with Hil but there are some wonderful phrases in the essay on the value of education – I do disagree with Friedman’s slippery slope argument and his argument that denationalisation automatically and magically equates to more choice, among many of his conclusions.) So, post-Friedman and Dawkins, we have a business model between educator and student, more fitting facilitator and consumer.

So, if competition increases choice, as Friedman asserts, has it? Hil refers (not too seriously) to specialist courses in courses that specialise in the study of Surfing, Casinos, David Beckham, Judge Judy, Cyberporn and the Phallus. (One can only hope that there are interlocking partial credit arrangements for the last two, if not four, given Rule #34.) Rather than an indicator of choice, this is an indicator of the “sexing up” of University as part of branding and consumerist issues. (Certainly, in Australia, we are seeing more schools and areas close than we are seeing open and there is no real sense of a locus of excellence for some areas outside of naming rights for institutions.)

In using business language, the University is implicitly stating that they understand the need to speak this language because it allows the consumer to attach value to our offerings but, because we aren’t really businesses, we come across as an amateur theatrical society – possibly looking good out the front but utter chaos behind. Part of this is the myth of our commitment to certain activities such as teaching, which is being increasingly carried out by casual staff, who may not have the background for the course or given the time to develop it. There is an expectation of expertise and deep familiarity, that we as teachers have thought about the work, imbibed it and considered it from all angles in order to move beyond understanding into wisdom – but this is too much to develop in one week! Hil does quickly note that he believes that vast majority of academics are doing the right thing but the increasing student numbers and class sizes, combined with an increasing ‘casualising’ of the work force, are taking us down a certain path.

Hil then talks about the implicit conflict of being a partially publicly, partially externally funded entity and that he believes that public funds were deliberately not given to Universities to make us more private and entrepreneurial. (I may have missed it but I don’t believe that he gives much evidence for this and the host did question him.) Private Universities are up front about being profit seeking, whereas we in the public sector walk a blurred line. Hil also feels pressure from directed retention policies to step attrition that, he feels, can compromise academics standards.

Academics are speaking out but mostly in private, Hil believes, because they are concerned for their futures and retribution. Academic tenure, the principle that academics can speak truth to power, is mostly eroded, with some institutions demanding clearance on all public statements concerning higher education and University matters. He’s also surprised that our Union membership, the National Tertiary Education Union, is very, very low, which appeared to surprise him given the level of discontent. On that note, the interview ended.

(From my perspective, an interesting interview but not a great many supporting facts but, given the topic, that’s probably understandable. I have already ordered Richard Hil’s book to have a look through and if I find something else useful, I’ll blog about it. I note that, at my University, we have very strict guidelines for assigning work to casual lecturing staff, including selecting an area of expertise, providing mentorship and training and most of what Hil talks about here does not appear to have taken as strong a hold in my University as other places, although I can definitely see tendrils!)

← Older Entries
Newer Entries →