Beating the Blank

I’m currently running one of my favourite teaching activities, the collaborative practical. My students are split into groups and are, as individual groups, trying to solve a small set of problems. The answers matter, but the process and group discussion is what I’m after.

Already, the person who has resisted all of my attempts to open up (for the last three days) is actively taking part, engaged and is contributing to their group. Other groups are discussing system aspects and, at times, having polite but intense arguments about interpretation.

Right now, I’ve stepped back to let them have a think, consolidate their ideas and let them start putting their notions down, to submit to me later.

Every student is currently engaged, everyone is contributing. There were some blank faces by the end of a 16 hour intensive weekend – all of those blank faces are gone.


Why am I here?

I’m currently in Singapore, teaching an intensive course as 3 hours Friday night, 6 hours Saturday and 7 hours Sunday. Obviously a course like this poses challenges for both staff and students, given the intensive nature, so I try to make it very interactive, full of peer activity and very little ‘just listening to Nick speak’. (Sometimes I succeed better than others.)

On Friday night, I wrote two lines up on the board:

“Why are you here?”

“Why am I here?”

My first discussion with the class, at 7pm on a Friday night after they’d been at work all day, was on these questions. As always, initial participation was guarded. Most students are in classes to meet requirements, pass exams and get pieces of paper – asking questions about this can cause some soul searching. So I switched to a discussion about why I was there.

Pause. Then a cautious suggestion.

“Money?” (Class laughter)

This gave me a more relaxed class to talk to and the chance to talk about all the reasons I could be there. After a minute or two’s (guided) discussion, I heard:

“You’re here to share your knowledge?”

Which then guided us to the next stage of the intro discussion – the important bit. The fact that if they know the work then passing the exam will be relatively easy. The fact that I care about what they know and that, when they leave, they should be able to practise their art with confidence.

The whole activity took about 5 minutes and set the tone for highly engaged discussion in and around 28 people, late on a Friday night, that happened to include a lot of information on Distributed Systems. It’s an ice breaker, a warm-up and it also tells the students what I need them to know: that I need them to know.


It’s Saturday, it must be Singapore

The next few messages have the theme of Intensive Teaching, as that’s what I’m doing this weekend, at our Singapore campus. I have 28 students, half a semester of content and 16 hours over three days. These students almost all have responsible full-time jobs and, in Singapore, that means 5.5 days a week.

My challenge is to present the material of Distributed Systems in an interesting, engaging and informative way to 28 people who, by Sunday, will be at risk of narcolepsy. Fortunately, the students are keen, the material is quite interesting by itself and I’ve done this before.

I hope that some of the posts that follow give you some interesting insights into a type of teaching that we don’t do all that often in higher ed – the ultra-intensive. I look forward to your comments.


False praise but I love your journal (random number)

You’ll excuse me writing a post that is reaction to my comment spam but a recent spate of spam reminded me of something important.

False or insincere praise is worse than no praise at all.

I use WordPress’s ARMY OF AUTOMATION to detect and corral my spam comments into an area where I can inspect them and delete them, without all of you having to read about false Google page rank updates, or notes saying things like “Wow, I loved this, keep it up 782346”. Things designed to take you to trap blogs or URLs to give some person somewhere some money. But I still manually review the spam to ensure that some poor ‘fan’ hasn’t written something that tipped the false positive scale. What this all reminded me of was my disappointment when I realised that something that I thought was personalised praise was actually automated nonsense, designed to suck me in. It’s deflating.

Some years ago, when I was just starting out, I received an e-mail from the chair of a relatively highly ranked conference inviting me to be on the program committee. Given that I’d published a bit in the area, I thought it was personal recognition and was really chuffed until my wife, very, VERY delicately, pointed out that it had gone to a giant mailing list called DBWORLD. (Yes, I should have noticed, but in those days I read the content before author and subject. These days, I do a lot more pre-filtering.) Every single person on that list had received that e-mail. This was a not a personal e-mail in recognition of any achievement on my part, it was a seat-filling exercise that had hit me as part of a wave.

There is a big difference between addressing small groups of people within the large, while openly admitting that you’re speaking in the large, and sending something that appears to be personal, but is secretly in the large. If you’re sending out messages of congratulations to the top performers in your class, why not spend the extra effort to send each person a message? They probably are going to talk about it and may notice how similar your message is – some may not care, of course, but some might. People who aren’t showing up? You can probably use blind carbon copy to send out the initial reminder, because it’s not SUCH a personalised message, and then personalise as they respond, or you can start personal, if you can manage it.

If you have a group of top students, but each one has nailed down a specific aspect of the course, noting the specific achievement as part of genuine and personal praise is, I suspect, going to have a far greater effect than a blanket e-mail saying “You’ve all done well”. “You’ve all done well” means I sorted the marks, selected the top 5 and pasted their e-mail into a mail merger. Specific praise, to reinforce that you have read it, you do know it, and you know at least some of that student’s mind, is going to reinforce the reality that you did mark it, what was submitted was noted, the work that went into it mattered.

This is what we’re saying when we produce genuine praise: “what you did mattered and it was good”

That’s why all feedback should be genuine and grounded in the work. Even if you’re giving general feedback to a class, I think it’s really helpful to find as much ‘resonant truth’ as possible – the feedback that everyone nods along to and goes ‘yes’. I could talk about authenticity and the importance of being genuine for hours but, once again, that’s another post.

Don’t get me wrong. Genuine praise and follow-up on the mass scale is better than none at all. If you have the time and resources, then we can probably all agree that personalised is better than general.

However, false praise, or insincere or misdirected praise, is counter-productive and really doesn’t have much place in our practices. It’s false and, ultimately, we’re about truth.

 


Everything can convey a message

While reviewing my blog, I noticed that the ‘site visitor’ graphics on the top left looked a little like a city skyline.

I don’t know which city is represented in a tiny set of pixels but I have the urge to click across it and explore it. To see what pops up.

Of course, the visual I see is not tied to the information overlay that I have placed upon it… but what a curious idea.

A tiny idea, visually engaging, to drag someone off into a world of information.


You Can’t Believe You Did So Badly?

A common e-mail comment I’ve had from students who didn’t get the mark that they were expecting is often “I can’t believe I did so badly” or words to that effect. “I knew I did well in the exam.” “I answered all of the questions correctly.” All of these are the same and, expect for the rare occasion when we have made an administrative error in assembling the mark, it usually indicates one thing.

The student doesn’t have enough knowledge to be able to assess their level of knowledge.

It would be glib to talk about Dunning-Kruger here, so I suspect it would be even more glib to mention it in passing and not discuss it! The Dunning-Kruger effect is a bias in your thinking that leads people with lower skill level to have an artificially high assessment of their own skills. The converse is that the highly skilled often underrate their ability.

When these students look at their exam scripts or assessments again, as I encourage them to do, the illusory has to confront reality. They see where I drew a red line across a page to indicate that they didn’t answer, say, question 2 at all, which cost them up to 30 marks. They see the places where I asked them to explain an unlabelled diagram, or to give me the detail underneath their broad brush statements. Sometimes, it’s seeing where I’ve written ‘sorry, no’  beside where they’ve answered both true and false to the same question. Without explanation. Then again, some students look at what I’ve said and try to find any possible wriggle room to get extra marks. Where it’s trying it on, it won’t get them anywhere, but I understand it. Where it’s a genuine lack of understanding as to why what they’ve provided is not a complete answer? That’s sadder, because they’re now functioning at a level where, until they advance, they will fail and not understand why they’re failing.

Worse, they may not realise even that they need help, or what kind it will take, to improve their skills. They’re fine – it’s the rest of the world, including me, that’s wrong.

You only have to speak to some students who have been around a little too long, or who left without their degrees, to realise that among those who have genuine problems, but are aware of them and striving to fix them, there are those who have equally genuine problems but assume that it’s nothing they can fix because it’s not their fault.

Does anyone out there have suggestions as to how we can help or approach these students, dealing with the delicate matter of their feelings as well as the cognitive bias? Please share them!


Failures of Politeness: Write Down Your Rules!

Being a University-level educator has some major advantages. One of the most significant is the degree to which the educators before me, in the school system, have shouldered the load of disruptive students, violent students – the students who didn’t manage to make it through the educational system and get into Uni. My heartfelt thanks, as always, for the effort and patience that it must take!

This gives me, 99% of the time, a very polite classroom. There’s always the impersonal indifference of late teen-age years (I remember how many other really important things I had to do, as well) but that’s usually fairly easily managed. It all runs like a large-ish dinner party with strangers most of the time. People pass the mustard when asked, generally share the food and no-one does anything diabolical. That’s why it’s surprising when you run across someone who breaks through that fragile and informal social compact. Let’s face it, you snatch the bread at home and someone’s going to have words with you. Steal the host’s fork and he or she will look at you with surprise, grab another one and, depending on what you do next, possibly etch your name into the ‘do not invite again’ column. Now that kind of ‘transgression’ is an obvious one – someone’s being rude, for whatever reason.

But let’s talk about finger bowls. A finger bowl is a bowl full of warm water with a squeeze of lemon in it. It’s used to wash your fingers when eating certain messy courses that must be handled manually. If that shows up on the table and you don’t know what it is, chances are you’re going to think it’s weak soup. There’s an etiquette in using these but any decent host is going to work out (a) if their guests know what these things are and clearly illustrate their use or (b) serve something else. Putting people in a situation where they accidentally break rules in ignorance  doesn’t do anything other than upset people.

Community discussion, such as public forums or debates, can be confusing for some students, especially when they don’t understand the implicit social compact in play. When unwritten rules do get broken, people’s feelings get hurt on both sides and it’s hard to know what to do. If someone stood up and shouted in one of my lectures, I’d eventually call security most likely but the disruption has happened and the damage is done. If you have to forcibly march someone out of a computer lab because they’re causing trouble, the lesson is shot for the next 10-15 minutes, no matter how much everyone present agrees with the action. We deal with most of the ‘extreme’ actions with a well-publicised list of acceptable behaviours, but what about the implicit ones? The subtle ones?

This is why I support writing all of the rules down. When you set up an electronic forum, I think it’s really helpful to have a list of (mostly) DOs and (a few) DON’Ts. (My pet theory is that people remember verbs, not modifiers. Hence PLEASE WALK is better than DON’T RUN because it puts the right action in your head. No doubt someone has proven or disprove this. Comments welcome!) If someone hits one of the rules, you can moderate in a transparent and fair fashion. You don’t set up trip lines for people to stumble into because then people will wonder where they can and can’t step – restricting motion when you want them to be embracing opportunity! Do you have in-class discussions? A simple set of guidelines can be put on a forum, or an A4 sheet, or as part of the moderator’s kit and everyone can be briefed.

Unwritten rules, like the arbitrary finer detail of politeness, can be confusing, divisive and, in an educational setting, are a hindrance rather than a help. You don’t know all of the details of your students’ background and, in an ideal world, everyone would be able to come here. So rather than depending on everyone in the world knowing the unwritten rules, let’s write it all down and avoid these unnecessary failures of politeness.


Who’s better, who’s best?

I’ve mentioned earlier that I assemble my final grades out of a number of assignments, all designed to test different areas, where a range of marks are possible. There are very few 0 or 1 situations in the assignments, with the possible exception of participation marks (and these never make up a big percentage of anything). Now the natural spread of marks across all of these assignments contributes to a slightly noisy final mark. As a general indicator of performance, the band into which your final mark falls is going to be pretty reliable. If you get 50-64 in one of my classes then you know enough to move on from the course. You have enough knowledge. If you get 45-49… well, then we offer you a chance to sit the exam again or, sometimes, do more assignment work, because we’re sitting in a slightly noisy zone and giving you another chance might get you over the edge.

But what is the real difference between a student who gets 50 and one who gets 60? 64 and 65? 98 and 99? Which of these leap out to you and say “Well, obviously, this student is better!” What do we even mean by better  in this case? In my opinion, if a course is designed correctly, and the students are properly prepared, then a pass mark is available for everyone if they do the work and apply themselves. The higher grades are only open to people who move on to synthesis, or can demonstrate deeper understanding, or can communicate their ideas more clearly, or… you get the idea. It’s often a sign of increased effort, to a large extent, than greater ability. (I’m sure we all have very smart students who are mildly bemused as they fail again because they just didn’t hand enough up or actually read a book.) In terms of general bracketing, a student who achieves a credit has probably done more than one who has a pass.

Okay, yes, the borders are tricky but with enough assessment work and opportunity, sometimes people just don’t make it over the line. If we have the ability to fail people, then we can apply some sort of higher level banding to higher level concepts. (There’s lots to say here and I’m trying to keep this short and on point, but I’ll try to return to this later.)

When it comes to numerical marks, though, inside the bands, that’s a bit shakier for me. Yes, there’s a difference between 100 and 85 (do we need a VHD?), but 99 and 98? No, they’re pretty much the same. It’s a shame, therefore, that we often rank students in our heads on these numbers rather than their grades. Or we average the grades and come up with a number that is as accurate and as useful as an average can be. Should we be using the average? Mode? Median? Harmonic mean? What makes sense in the face of what these students have done?

It’s a shame then that values like these, or averages of these, or GPAs, have so much weight in the community. It’s not as if most people dig down, or have the ability to dig down, into the underlying courses to see what these things actually mean. Perhaps we should be awarding marks for mastery of core materials, a separate mark for performance in projects and yet another mark for advanced or extension work, tying it into industry practices and terminology? Then you could look at the numbers, as they are, and having isolated the components that have been jammed together before, get an idea of what that number means.

Imagine that a student has a perfect GPA for a set of courses that, as it turns out, have nothing to do with what your company does – except that the name sounds about right. Another student with a slightly worse GPA has perfect marks for core, including the language and techniques you want, and is fantastic on projects. You may find this out at interview or in the application but, if you only take perfect GPAs and ditch everything else, you’ll never even see number 2.

I’m not saying that our current system is unworkable, because it obviously works, but I have started to wonder what else we could do – in an ideal world. There’s no solution here, this is not amazingly pragmatic thinking, which doesn’t take into account the fact that so many people who take our students want that number. But that doesn’t mean we can’t talk about it and think about it. Maybe there’s something better? The first step is to work out what it is.


I ric, you ric, we all ric for… rubric?

In teaching, we use rubrics to let students know what our expectations of quality are. I can set an assignment, list my expectations in order to achieve a certain grade and, better still, I can then mark against the same statements. Clear, fair, transparent. Here’s an example for, say, an essay on developments in distributed systems in the early 21st Century. I don’t have access to some of my previous rubrics at the moment so this is a little ‘off the top of my head’. Please focus on the intent (conveying a specific requirement to the student) rather than and/or arrangement of clauses.

References:

  • An unsatisfactory grade will be awarded for references if you have fewer than 5 relevant references or the references are not primary sources or high quality peer-reviewed publications. You should not include unreviewed work, Wikipedia entries (unless to illustrate a specific technical point that deals with Wikipedia or Wikipedia policy) or Personal Communications (unless you clear these with me first). To determine the ‘ranking’ or ‘quality’ of a publication, you may use Impact Factors or CORE/ERA rankings. If in doubt as to the nature of the publications or the quality of your references, check your references with me by posting to the main forum’s Student Questions section.
  • To achieve a satisfactory mark, you must provide clear and accurate citations, at least 5 as described above, for all external material included in your work (see (web page reference) for University citation standards and avoiding accidental plagiarism).
  • To achieve a mark in the range of credit or distinction, your clear and accurate citations should include at least 10 relevant references from A or A* journals and conferences.
  • To achieve the best mark, your references should also reflect a thorough reading of the area, with at least 20 relevant references,  10 derived from A* journals and the remainder A/A* publications.

It’s fairly easy for a student to work out what they need to do in order to pass, do better and excel. It’s also fairly easy for me to mark because I know where the core reference should be coming from and, if one appears out of left field, then the worst case situation is that I learn about a new venue.

I would usually arrange this in tabular fashion so that all of the requirements for the essay were arranged along the left column and, as you moved across the row, you could look up to column top and see Unsatisfactory, Satisfactory and look in the intersection of row and column to find out what you have to achieve in aspect X to achieve outcome Y. There is still a great deal of room for movement in marking this but, as a framework, I find it useful.

Let me compare this with something I received once for an assignment.

References: Your references should be good, with enough to support your argument. Remember to cite correctly. We take plagiarism very seriously.

I leave it to the reader to decide which of these tell them what they have to do in a more useful way, and which they think would probably be easier to mark in a fair and consistent manner.


Those Dang Kids and Their Phones, Computers and iPads.

Some my students use computers in my lectures and, while I’m sure that a number of them are writing notes or looking at the notes, I’m sure that many are ‘multi-tasking’ with social media or the like. They’re not playing games, because I know what they looks and sounds like (wwwwwwwddaaaaa*click**click*wwww *DAMN*) but these mobile platforms are more powerful and more pervasive so this will happen a lot. Now, despite the presence of computers, I can usually get the students with gadgets to participate, answer questions and do all of those good things. If I have a question, I don’t have to repeat the previous 5 minutes of speaking to get an answer. They’re mostly listening. But, whenever the gadgets are out, there’s always a worry that those dang kids are surfing the internet rather than listening to you.

Now, I’m sure that some of you have very strong opinions about this. Why do students do this given they’ve bothered to show up?

I have, in recent experience, attended meetings which had an obligatory nature to them and, among my colleagues, seen a lot of iPads, notebooks, laptops and phones in use during these meetings. Now you’d expect that someone would be most active on this mobile reference and presentation platform when it was their turn to talk or be talked to, but generally it seemed to be when the person in question wasn’t required to do anything. In fact, these gadgets get the most workout when the person holding them hasn’t done anything in the meeting nor is expected to. Armed with a gadget, these people (who may have been colleagues) had tuned out because the major reason that they were in the meeting was because someone had told them that they had to be. If they didn’t have gadgets they’d probably be sleeping (quietly), doodling or talking to the person next to them.

Every time someone complains about students who focus on the gadgets rather than the lecture, I think about those meetings where I’ve checked my mail during a dull patch in a meeting. Or done some other (useful and real) work that had nothing to do with the event that I was in.

I know, I’m a terrible human being with a short atten… what? But the thing is, give me a reason that I should be somewhere and a reason to participate, interact and listen? I’ll be there, armed with my knowledge and my supporting fleet of tech. THEN, the gadgets are only used as they should be. I don’t doodle. I stay focused. No doubt everyone reading this has a better attention span than I do and has never, ever been bored in a compulsory meeting on new OH&S standards for refrigerator cleanliness or has tuned out while trying to listen to someone reading tiny-font info-dense slides to you, regarding new research standards.

Hmmm.

We want our students to be like us. In many ways they already are. Yes, some students are very hard to engage but, for most of them, if they have a reason to be there and a reason to listen and take part, chances are they will.