Workshop report: ALTC Workshop “Assessing student learning against the Engineering Accreditation Competency Standards: A practical approach”

I was fortunate to be able to attend a 3 hour workshop today presented by Professor Wageeh Boles, Queensland University of Technology, and Professor Jeffrey (Jeff) Froyd, Texas A&M, on how we could assess student learning against the accreditation competency standards in Engineering. I’ve seen Wageeh present before in his capacity as an Australian Learning and Teaching Council ALTC National Teaching Fellowship and greatly enjoyed it, so I was looking forward to today. (Note: the ALTC has been replaced with the Office for Learning and Teaching, OLT, but a number of schemes are still labelled under the old title. Fortunately, I speak acronym.)

Both Wageeh and Jeff spoke at length about why we were undertaking assessment and we started by looking at the big picture: University graduate capabilities and the Engineers Australia accreditation criteria. Like it or not, we live in a world where people expect our students to be able to achieve well-defined things and be able to demonstrate certain skills. To focus on the course, unit, teaching and learning objectives and assessment alone, without framing this in the national and University expectations is to risk not producing the students that are expected or desired. Ultimately if the high level and local requirements aren’t linked then they should be because otherwise we’re probably not pursuing the right objectives. (Is it too soon to mention pedagogical luck again?)

We then discussed three types of assessment:

  • Assessment FOR Learning: Which is for teachers and allows them to determine the next steps in advancing learning.
  • Assessment AS Learning: Which is for students and allows them to monitor and reflect upon their own progress (effectively formative).
  • Assessment OF Learning: Which is used to assess what the students have learned and is most often characterised as summative learning.

But, after being asked about the formative/summative approach, this was recast into a decision making framework. We carry out assessment of all kinds to allow people to make better decisions and the people, in this situation, are Educators and Students. When we see the results of the summative assessment we, as teachers, can then ask “What decisions do we need to make for this class?” to improve the levels of knowledge demonstrated in the summative. When the students see the result of formative assessment, we then have the question “What decisions do students need to make” to improve their own understanding. The final aspect, Assessment FOR Learning, is going to cover those areas of assessment that help both educators and students to make better decisions by making changes to the overall course in response to what we’re seeing.

This is a powerful concept as it identifies assessment in terms of responsible groups: this assessment involves one group, the other or both and this is why you need to think about the results. (As an aside, this is why I strongly subscribe to the idea that formative assessment should never have an extrinsic motivating aspect, like empty or easy submission marks, because it stops the student focussing on the feedback, which will help their decisions, and makes it look summative, which suddenly starts to look like the educator’s problem.)

One point that came out repeatedly was that our assessment methods should be varied. If your entire assessment is based on a single exam, of one type of question, at the end of the semester then you really only have a single point of data. Anyone who has ever drawn a line on a graph knows that a single point tells you nothing about the shape of the line and, ultimately, the more points that yo can plot accurately, the more you can work out what is actually happening. However, varying assessment methods doesn’t mean replicating or proxying the exam, it means providing different assessment types, varying questions, changing assessment over time. (Yes, this was stressed: changing assessment from offering to offering is important and is much a part of varying assessment as any other component.)

All delightful music to my ears, which was just was well as we all worked very hard, talking, discussing and sharing ideas throughout the groups. We had a range of people who were mostly from within the Faculty and, while it was a small group and full of the usual faces, we all worked well, had an open discussion and there were some first-timers who obviously learned a lot.

What I found great about this was that it was very strongly practical. We worked on our own courses, looked for points for improvement and I took away four points of improvement that I’m currently working on: a fantastic result for a three-hour investment. Our students don’t need to just have done assessment that makes it look like they know their stuff, they have to actually know their stuff and be confident with it. Job ready. Able to stand up and demonstrate their skills. Ready for reality.

As was discussed in the workshop, assessment of learning occurs when Lecturers:

  • Use evidence of student learning
  • to make judgements on student achievement
  • against goals and standards

And this identifies some of our key problems. We often gather all of the evidence, whether it’s final grades or Student Evaluations, at a point when the students have left, or are just about to leave, the course. How can we change this course for that student? We are always working one step in the past. Even if we do have the data, do we have the time and the knowledge to make the right judgement? If so, is it defensible, fair and meeting the standards that we should be meeting? We can’t apply standards from 20 years ago because that’s what we’re used to. The future, in Australia, is death by educational acronyms (AQF, TEQSA, EA, ACS, OLT…) but these are the standards by which we are accredited and these are the yardsticks by which our students will be judged. If we want to change those then, sure, we can argue this at the Government level but until then, these have to be taken into account, along with all of our discipline, faculty and University requirements.

I think that this will probably spill over in a second post but, in short, if you get a chance to see Wageeh and Jeff on the road with this workshop then, please, set aside the time to go and leave time for a chat afterwards. This is one of the most rewarding and useful activities that I’ve done this year – and I’ve had a very good year for thinking about CS Education.


Offering a Way Forward: Saying “No” Constructively

I was a shockingly bad undergraduate student. I am rather ashamed of this, in hindsight, because it could have been the most dreadful squandering of the opportunities that had been made available to me. Fortunately, intervention at a later time allowed me to recover from this mis-step and I arrived at the lofty heights that you know now: a semi-anonymous blogger with logorrhoea, employed as a low level academic at a good University. 🙂

One of the experiences that sticks in my mind, and I believe I’ve mentioned it before, was a point when I was refused supplementary examination for a course. It was, to be honest, completely reasonable that my request was refused as I had achieved the lofty score of zero for that examination and, although I was sick on the day, I had done no work at all during that term. There is no argument that my request was anything other than a small part in my continuing quest to scrape by and do as little as possible. However, what I will address is what happened next.

Nothing.

I was in an academic’s office, I had scored zero on the exam and I had not done any work. If someone showed up in my office under those conditions, I’d at least ask “Why?” Ok, sure, you’re not getting a supp from me, because I’m not a soft touch by any stretch of the imagination, but I can’t help feeling that I’m just not doing my job unless I work out what has gone on and whether I need to step in or refer you to someone else. This probably explains why I’m so prone to jumping up and down when people talk about bad students, as if this is a permanent and immutable condition, because while I completely accept that some students (a small group) will resist all efforts to bring them into a culture, I strongly believe that it is far more likely that little effort has been spent to try and recover these students in the first place.

Was the academic right to refuse my application? Yes, no argument, but the methodology sucked. No follow-up. No identification of problems. Did I learn from this? No, I didn’t. I distanced myself from the mark because I had expended no effort – this may sound familiar from my discussion of pedagogical luck from yesterday. The same self-destructive patterns played out until someone (a big group of someones actually) took enough interest in me to make me explain what I was doing, make me realise what I was doing and show me the way forward. (And, for the record, thank you to all my friends and my wife, Katrina, but I’d like to specifically note Cathy and Andrew, where poor Cathy had to basically help me focus while I tried to learn what being a real student was about. And thanks to KJM for suggesting that I start doing the new degree in the first place.)

The reason that this is back in my mind is that our Summer Research Scholarships are being advertised at the moment, where students come and work on a research project for 6 weeks over Summer with a small stipend. Competition for these scholarships is surprisingly fierce and I am lucky enough to be offering some interesting projects, both in networking and education research, that people find attractive. Most of the students who have applied for my support have had a level of interest and academic achievement that I can easily support them in what is a highly competitive environment (University ranking systems, not mine, I hasten to add). However, one of the applicants was not all that competitive, so I had to say that I really couldn’t support them in their application. Yet. But here are some ways to be ready for this next year. What was the problem for this year? How are you going at the moment? Ok, keep working on this and I look forward to seeing you come back to me next year, but here’s another suggestion.

The way of saying “No” is important, and I believe that this student can reach the required level, they’re just not there yet. So we discussed how they could improve their overall eligibility, including coming up with their own ideas to fit in with their thoughts on research, so that when they come back next year, they come back not only more eligible but with a strong idea of what they want to do with us. I expect to see this student back next year because I’ve done similar things before and, more often than not, the student comes back: purposeful, more focused and much, much more ready to undertake the kind of projects involved. I have already loosely allocated some resources in case the student does what I’ve suggested for improvement in the shorter term. Next time, I hope that I’m in a position where I must support this student’s application but I can only claim to have helped here if I’ve taken the time to think about what is going on and how I might be able to help.

In the spirit of charity, I must confess that I have no idea if the lecturer, who didn’t ask me about what I was up to, was just having a bad day or if my performance was so poor that he just decided that I was one of the small number who wasn’t ever going to come good. You can’t judge someone on a single event or interaction and it would be ridiculous to not admit to the human failing of the ‘bad day’. However, my overall experience, as a bad student, was that I was not worth wasting time on. I was not worth the risk. And, yet, no-one ever asked me if there was a reason for it or why I was doing it, or if there was something that I needed help with.

I’m happy to say that I believe that a culture like that is now, rightly, judged and found wanting. That people are far more likely to ask why and suggest how or who to see next. But it’s not universal. We may not always achieve universality, people are people and bad days are bad days, but the first step, for me, is looking at how we say “No.”

 


Brief Stats Update: I appear to have written two more books

On May 6th, I congratulated Mark Guzdial on his 1000th post and I noted that I had written 102,136 words, an average of 676 words per post, with 151 posts over 126 days. I commented that, at that rate, I could expect to produce about 180,000 more words by the end of the year, for a total of about 280,000. So, to summarise, my average posting level was at rate of 1.2 posts per day, and 676 words per post.

Today, I reanalysed the blog to see how I was going. This post will be published on Tuesday the 9th, my time, and the analysis here does not include itself. So, up until all activity on Monday the 8th, Central Australian Daylight Saving Time, here are the stats.

Total word count: 273,639. Total number of posts: 343. Number of words per post: 798. Number of posts per day: 1.23. I will reach my end of year projected word count in about 9 days.

I knew that I had been writing longer posts, you may remember that I’ve deliberately tried to keep the posts to around 1,000 where possible, but it’s obvious that I’m just not that capable of writing a short post! In the long term, I’d expect this to approach 1,000 words/post because of my goal to limit myself to that, with the occasional overshoot. I’m surprised by the consistency in number of posts per day. The previous average was a smidgen under 1.2 but I wanted to clarify that there has been a minor increase. Given that my goal was not to necessarily hit exactly 1/day but to set aside time to think about learning and teaching every day, I’m happy with that.

The word count, however, is terrifying. One of the reasons that I wanted to talk about this is to identify how much work something like this is, not to either over inflate myself or to put you off, but to help anyone out there who is considering such a venture. Let me explain some things first.

  1. I have been typing in one form or another since 1977. I was exposed to computers early on and, while I’ve never been trained to touch type, I have that nasty hybrid version where I don’t use all of my fingers but still don’t have to look at the keyboard.
  2. I can sustain a typing speed of about 2,500 words/hour for fiction for quite a long time. That includes the aspects of creativity required, not dictation or transcription. It is very tiring, however, and too much of it makes me amusingly incoherent.
  3. I do not have any problems with repetitive strain injury and I have a couple of excellent working spaces with fast computers and big screens.
  4. I love to write.

So, I’m starting from a good basis and, let me stress, I love to write. Now let me tell you about the problems that this project has revealed.

  1. I produce two kinds of posts: research focused and the more anecdotal. Anecdotal posts can be written up quickly but the moment any research, pre-reading or reformulation is required, it will take me about an hour or two to get a post together. So that cute high speed production drops to about 500-1000 words/hour.
  2. Research posts are the result of hours of reading and quite a lot of associated thought. My best posts start from a set of papers that I read, I then mull on it for a few days and finally it all comes together. I often ask someone else to look at the work to see how it sits in the queue.
  3. I’m always better when I don’t have to produce something for tomorrow. When the post queue is dry, I don’t have the time to read in detail or mull so I have to either pull a previous draft from the queue and see if I can fix it (and I’ve pretty much run out of those) or I have to come up with an idea now and write it now. All too often, these end up being relatively empty opinion pieces.
  4. If you are already tired, writing can be very tiring and you lose a lot of the fiero and inspiration from writing a good post.

I have probably spent, by all of these figures and time estimates, somewhere around 274 hours on this project. That’s just under 7 working weeks at 40 hours/week. No wonder I feel tired sometimes!

I am already, as you know, looking to change the posting frequency next year because I wish to focus on the quality of my work rather than the volume of my output. I still plan to have that hour or so put aside every day to contemplate and carry out research on learning and teaching but it will no longer be tied to an associated posting deadline. My original plan had an output requirement to force me to carry out the work. Unsurprisingly, oh brave new world that has such extrinsic motivating factors in it, I have become focused on the post, rather than the underlying research. My word count indicates that I am writing but, once this year is over, the review that I carry out will be to make sure that every word written from that point on is both valuable and necessary. My satisfaction in the contribution and utility of those posts I do make will replace any other quantitative measures of output.

My experience in this can be summarised quite simply. Setting a posting schedule that is too restrictive risks you putting the emphasis on the wrong component, where setting aside a regular time to study and contemplate the issues that lead to a good post is a far wiser investment. If you want to write this much, then it cannot be too much of a chore and, honestly, loving writing is almost essential, I feel. Fortunately, I have more than enough to keep the post queue going to the end of the year, as I’m working on a number of papers and ideas that will naturally end up here but I feel that I have, very much, achieved what I originally set to to do. I now deeply value the scholarship of learning and teaching and have learned enough to know that I have a great deal more to learn.

From a personal perspective, I believe that all of the words written have been valuable to me but, from next year, I have to make sure that the words I write are equally valuable to other people.

I’ll finish with something amusing. Someone asked me the other day how many words I’d written and, off the top of my head, I said “about 140,000” and thought that I was possibly over-claiming. The fact that I was under claiming by almost a factor of two never would have occurred to me, nor the fact that I had written more words than can be found in Order of the Phoenix. While I may wish to reclaim my reading time once this is over, for any fiction publishers reading this, I will have some free time next year! 🙂


Beautiful Corrections

(Sorry about the delay in today’s post. Yesterday afternoon, I took an early minute, and my wife and I went to view Australian Aboriginal art at the Tandanya National Aboriginal Cultural Institute in Adelaide, had a drink in a pub and then had a long and relaxing dinner at a local ethically-sourced Italian restaurant with a wickedly good pizza oven and a great Langhe Nebbiolo. This didn’t leave much time for blogging. Work/life balance-wise, however, it was a winner.)

Yesterday, I referred to an article on New Dorp high school and I wanted to bring out one of the other things that I really liked about their approach to a ‘get the kids writing’ program. As the article says, thinking, reading and speaking are all interconnected and are reinforced through sound instruction in good writing. This immediately leads to the conclusion that teaching people to write is going to lead to improvements across the board and, as a Probationary-plated social constructivist, I immediately think about constructive interaction between students based on great confidence in speaking, fuelled through a greater depth of understanding and ability to express your ideas.

The article discusses this, because classroom discussion became an opportunity for students to listen, think and be more precise in the way that they discussed their ideas. This can be a trap, as most educators know far too well, if students feel that they have to say something rather than that they have to say something that they can defend, explain or shows signs of reflection. (We see this in writing, too. “What I did on my holiday” is a relatively unassailable personal anecdote with no great guarantee of depth or need for defensible statement, yet “What was the most useful thing that you did on your holiday?” requires thought, comparison, reflection and review. To a degree, obviously. I’m not going to start early writers on a detailed comparison of Yves Klein blue and its apparent lifting from Picasso…)

It is very easy to take classroom discussion in the wrong way. You don’t always have to be cheerleadingly positive (warning: not a real adjective), but framing a critique or a question makes a big difference when you want to encourage discussion and build confidence. That’s why I like what I’m reading about in New Dorp (and I’ve seen elsewhere to a lesser degree), in that the students have a poster at the from of the class that lists ways to respond. For example:

  • I agree/disagree with ___ because …
  • I have a different opinion …
  • I have something to add …
  • Can you explain your answer?
  • I agree with ___ but I disagree with your conclusion (because) …

This is a far cry from the passive responses to a tired questioning approach of “Now, hands up if you think that John is correct”. With this framing, students are encouraged to contribute, contest and expand, but using a formal approach to the argument that reduces dependency upon ad homimen or genetic fallacy issues: we have to address what was said rather than the person or the group that it came from. It’s very easy to say “You’re wrong” or “That’s stupid” and it’s an easy answer that completely undermines what the faculty at New Dorp are trying to achieve.

It’s easy to see how this approach is useful in the higher educational sphere, especially once we get into student-based activities, because we can’t always be the facilitators ourselves, so the training of our sessional staff becomes crucial. One challenge for our sessional staff is how to respond to questions without ending up giving the answer away immediately or doing the work for the student. We expend a lot of time on training (Katrina does a great deal of work in this area) and this simple set of guiding questions and framing, as a training device for our staff as well as a template for our students, will allow us to keep the important lessons fresh and in everyone’s mind. We focus a lot on Contributing Student Pedagogy (CSP), a pedagogy that encourages students to contribute to other students’ learning, including valuing other contributions, generally using a high degree of role flexibility (sometimes you lead, sometimes you support and sometimes you organise, for example). We have a paper in the upcoming special issue of Computer Science Education on CSP, where we talk about this at length, but a simple semi-formal structuring of questions to assist people in thinking about how they are about to contribute or evaluate someone else’s contribution is a valuable component of this kind of approach.

To return to what New Dorp is attempting to do, these questions encourage all participants to think about the why and the because and how their contribution will work in with what has already been said. However, and this is non-trivial, having a semi-scripted start to a response also encourages the correct use of language, familiarity with key phrases and the correct use of modifiers and conjunctions. One of the issues identified at New Dorp was that poor writers couldn’t pull a U-turn in a sentence with much success. Although, despite, and words like that were effectively a mystery – sentences had to be artificially short, tightly focussed and lacking in complexity. Such a limitation greatly limits the degree of expressiveness available to the writer. Sentences don’t have to be long, but they have to be long enough. Sentences don’t have to contain long words, but they have to contain the right words. Ideas need to be expressed in a way that makes them easy to understand but this requires practice, practice and even more practice.

The script on the poster at the front is not a rigid proscription. The poster doesn’t say “Explain the use of adjectives in the sentence.” Instead, it provides a hook that a student can hang their own ideas upon, the leading sentence that starts the invasion of text into the bleak white space of a new page. It encourages discussion, support, interaction and the development of thought.

It appears that New Dorp’s approach is working. Students are improving. Students can write. Students can communicate their thoughts to other people successfully. They can use language. What a great improvement!


The Reputation of Australian IT IS Enhanced: A Closing Point

Professor Gernot Heiser has released the second part of his response to my blog post and the originating Australian IT article, and you can find it here. On reading both parts, I have amended my original posts to include links to Gernot’s responses, because they address both of the key questions from the original posts: by identifying why a process like this would be followed and how national benefit is served by it. The most important point to realise is that NICTA still owns a great deal of associated work from this project – rather than the cloak-and-dagger fire sale that was alluded to in the newspaper piece.

You can find the whole discussion still on my blog as I feel that the ongoing and evolving discussion illustrates one of the key advantages of the new technological models that we have: the ability to exchange ideas, update our published text and construct more accurate representations of knowledge. I would like to thank Professor Heiser for his responses, especially as it would have been very easy to either ignore them or be very dismissive. Instead, he’s provided a great deal more information that has certainly informed me as to how these decisions are made and what they mean.

I hope that you have enjoyed reading this as well, and that you have found the whole exchange useful.


Educating about Evil

While we focus on our discipline areas for education, we can never lose sight of the important role that teachers have in a student’s life. As I’ve said (in ones way or another) repeatedly, we have large footprints and a deep shadow: thinking that we are only obliged to worry about mathematics or the correct location of the comma is to risk taking actions that have a far greater impact than intended.

This is why I have no time for educators who sleep with their students, because they have reduced anything positive or supportive that they ever said to the student into a part of the seduction and it contaminates the relationship that the student will have with authority, possibly for the rest of their life. In the strongest terms I condemn this, not the least because it is almost always illegal, immoral and wrong, but because it is, at its heart, unscholarly, unthinking and anti-educational. If you want to teach, then you’ve put yourself in a position where your voice is going to carry more weight – and this brings responsibilities. Naively enough, one of the key responsibilities for me is that we must think carefully about our actions so that, by our thoughtless action or inaction, we do not facilitate evil.

I do not have a belief system that gives me a convenient Devil so, for me, evil is a concept that is very abstract, but no less real for not having a trident and cloven hooves. I know it when I see it. I know it when I see its hand at work and it is the shape of evil’s hand that I generally discuss with my students. Let me show you.

Elizabeth Eckford, girl in dark sunglasses, attempting to enter Central High in the Little Rock School District. (Photo: Will Counts)

Can you see it? Let me show you from another angle.

The girl screaming racist abuse is Hazel Massery. The year is 1957. (Photo: Will Counts)

That’s a 15 year old girl standing at the front who is, under established legal precedent, trying to enter a previously all-white school. The girl behind her, about the same age, is yelling this: “Go home, n____! Go back to Africa.” Those soldiers you see are national guardsman, stationed not to help an isolated 15 year old girl but, instead, to keep her and the other 8 students who haven’t shown up today, from bringing their black selves into this white classroom.

I see the hand of evil all over this incident but, via these photographs, but I see it most with its scaly digits clutched around Hazel Massery’s mouth. She had a family with troubles and a background entrenched racism, and Hazel was a troubled girl but, in this moment, she was a hysterical, screaming puppet, baying for the blood of a 15-year old girl who was just another human being. The people around Elizabeth are yelling “Lynch her!” “Drag her away!” Women who look like your grandma are spitting on her. But look at Hazel Massery. It’s hard to find a more spectacular example of the evil of the mob than this?

Fifty-five years ago this month, nine students tried to make it into the school and, finally, after being turned away three times by National Guardsman, they managed to enter, escorted by soldiers of the 101st Airborne. Some of the people in that crowd, smiling, chuckling, taking pictures – they are teachers. Elizabeth’s ongoing problems at school, and they were many, included one teacher who would not even take anything directly from Elizabeth’s hands because of the colour of Elizabeth’s skin. Elizabeth was systematically abused, isolated and bullied up until the time that the school got closed and she had to try and complete her studies by herself.

After months of abuse, one entry from Elizabeth’s experience reads: “She said that except for some broken glass thrown at her during lunch, she really had had a wonderful day.”

When I was a teenager, I attended a talk where a minister said that he had always expected the test of his faith and integrity to be a suave man with horns and a tail, wearing a good suit, who offered him a dollar to smoke a cigarette and spit on the Bible. As he got older, he realised that evil, in many forms, was much harder to recognise and, of course, that sometimes doing nothing counted as evil, if you didn’t take the opportunity to do good. (As I later realised, in the style of Edmund Burke!)

You know, I don’t expect that much of a lot of people, if no-one has gone to the trouble to actually educate them and shake those xenophobic beliefs that seem to accumulate when we’re in small, scared bands and huddled in the dark. But I do expect a great deal of anyone who takes up the role of educator. I expect them to stand up for the truth. To go looking for the light if they realise that they’re in the dark. To treat all students as what they could be rather than what they have been assumed to be.

But, base level, in any activity regarding students and mobs, formed from stupidity and bigotry, I expect the teachers to be in a circle around the students and facing out, standing between the mob and their charges, certainly not facing in and taking part in the discrimination.

There’s a Vanity Fair article where you can read more about this. We have, I am thankful to say, come a very long way but it is quite obvious that there is still some way to go. The VF article talks a lot about the good people of the community, who stood up, who helped up, who realised that this was wrong and you should read it because it is a story of hope. But let us never lose sight of what evil looks like, because I need to train my students to see it so that they can stamp on it.

Push it back into the darkness where it belongs and blind it with truth, facts, science, reasoning, enlightenment and goodness. In 100 years time I want someone who sees that picture to not even be able to understand why this would have happened. I want cute kids to cock their heads to one side and look confused – because they can see an obviously visual difference but not inequality or divide, and hence not establish it as in immutable categorical statement of worth or ability. I think we’re all part of the glorious pathway that will lead to that great and wonderful time. Naive? Yes. But we have to start somewhere and, fifty-five years ago, Elizabeth and eight other brave young people did just that. We’re just carrying it on.


Six ‘Easy’ Pieces? Richard Feynman and the Undergraduate Lectures

Richard P. Feynman was a Nobel prize winning theoretical physicist, who made great contributions to physics and the popularity of physics through his books and lectures. Among many other useful activities he developed Feynman diagrams, which provided a useful pictorial abstraction of the rather complicated mathematical expressions that govern the behaviour of subatomic particles.

Trust me when I tell you that this is easier to understand than the expressions.

This is a great tool in many ways because it makes the difficult more easy to understand, the abstract able to be represented in a (closer to) concrete manner and, above all, humans like pictures. Feynman was very interested in teaching as well because he felt that students could offer inspiration and because teaching could be a diversion when the well of theoretical physics creativity was running dry.  He was an opponent of rote learning and any approach to teaching that put the form before the function. He loved to explain and felt a strong duty to explain things clearly and correctly, with an emphasis on a key principle that if he couldn’t explain it at the freshman level, then it wasn’t yet understood fully.

In the 60’s Feynman was asked, by Caltech, to reinvigorate the teaching of undergraduates and, three years later, he produced the Feynman Lectures on Physics. I’ve read these before (I used to study Physics – I know, I seem so nice!) and so have many other people – it’s estimated that more than 3 million copies have been sold in various languages. I picked up a copy of the ‘cut-down’ version of the lectures “Six Easy Pieces”, recently re-published in Penguin (AU$ 9.95! Hooray for cheap books!)

Reading the 1989 Special Preface to the original lectures, re-printed in “Six Easy Pieces”, a strange fact emerges, which is that Feynman’s lectures did not necessarily succeed for their target audience, the undergraduates, but instead served to inspire the teachers. As Goodstein and Neugebauer noted, while the class started with 180 undergraduate students, many of the students dreaded the class and, over time, dropped out. While the class remained full, it was because of the increased occupation by faculty and graduate students.

In the original preface, by Feynman, he appears to have noticed that something was amiss because he reflects on the fact that he didn’t think it was a great success. One problem was that there was no feedback from the students to him to tell him how he was doing, whether they were keeping up. (Feynman provided very little outline and all of the homework assignments were created by other professors sitting in the class, furiously noting what had been covered and then creating the other work for recitation.) Feynman’s aim was to challenge and interest the best and brightest, he sought to not only direct the lecture at the smartest in the room  but to present work so that even the most brilliant in the room would be unable to cover it all. Feynman’s preface contains terms such as ‘sufficiently clever’, which may seem fine to some but to me indicate clearly that he, an astoundingly smart and still empathic human being, had at least an inkling that something had gone wrong between his vision and what happened in the classroom.

At the end of the preface, Feynman reflects, in a rather melancholy tone, “I don’t think I did very well by the students”. He is concerned that, based on the way that the the students handled the questions in the examination, that the system is a failure. A colleague points out that maybe 12-24 students appeared to really get it but you don’t have to be a very good mathematician to release that 24/180 (a nudge over 13%) is not the best rate of transfer. As Feynman gloomily responds (quoting Gibbon):

“The power of instruction is seldom of much efficacy except in those happy dispositions where it is almost superfluous”

Feynman finishes, with his characteristic insight, that the direct individual relationship between student and teacher is paramount, where the student discusses things and works with, and discusses, ideas. That it is impossible to learn very much by sitting in a lecture. But he sees himself torn between what he sees as the right way to proceed and the number of students that we have to teach.

And, 49 years later, we, the inheritors of Sisyphus, are still trying to push that same rock up the same educational hill. Richard Feynman, a grand communicator and superlative thinker and scientist, tried his hardest to make the lecture work and even he couldn’t do it. He had mountains of support and he was unhappy with the result. He is clearly articulating all of the ideas for which we now have so much evidence and, yet, here we still are with 1000-person lectures and students who might be able to plug some numbers into formulas but don’t necessarily know what it means to think inside our discipline or discuss ideas in a meaningful dialogue.

From a personal perspective, Feynman’s Lectures on Physics are one the reasons why I gave up physics. I was struggling to see how it all fitted together and I went to seek help. (I was also a terrible student in those days but this was one of the rare occasions when I tried to improve.) One of my lecturers told me that I should read Feynman’s lectures and because it was designed for undergrads, if I couldn’t get that, I wouldn’t be able to catch up – basically, I didn’t have the Physics brain. I read it. I didn’t get it. I sorted the world into “physicists” and “non-physicists”, with me in the second group. (This is probably not a bad outcome for the physics community and, years later, while I can now happily read Feynman, it certainly doesn’t excite me as much as what I’m doing now.) I imagine that Feynman himself, while not lamenting me leaving the field, would probably be at least mildly perturbed at such a weaponisation of his work. From reading about him, his books and prefaces, I believe that he expected a lot of his students but he never actually wanted to be unpleasant about it. His own prefaces record his unease with the course he produced. He has no doubts about the physics and the aim – but his implementation was not what he wanted and not what he believed to be the best approach.

So, when someone questions your educational research supported ideas for improving learning and teaching, grab a copy of “Six Easy Pieces” and get them to read all of the preface material. Feynman himself regarded a lot of areas in educational research as cargo cult science, which applies as well to any poorly constructed scientific experimentation, but it is quite obvious that on at least some of the most important issues regarding knowledge transfer, he had a deep understanding and commitment to improvement, because of his direct experience with undergraduates and his ability to openly criticise himself in order to improve.


Our Obligations: Moral and Legal?

Mark Guzdial raises an interesting point over at a BLOG@CACM article, namely that, if we don’t keep up to to date with contemporary practice in learning and teaching, can be considered unprofessional or even negligent or unethical? If we were surgeons who had not bothered to stay up to date then our patients, and certifying bodies, would be rightly upset. If we are teachers – then what?

The other issue Mark discusses is that of the legal requirement. The US has Title IX, which should extend the same participation rights to all genders for any education program or activity that attracts federal funding. If we do not construct activities that are inclusive (or we design activities that, by their nature, are exclusive) would we be liable under US law?

Mark’s final question is: If we know a better way to teach computing, are we professionally (and even legally) required to use it?

That is a spectacularly good question and, of course, it has no easy answer. Let me extend the idea of the surgeon by building on the doctors’ credo: primum non nocere (first, do no harm). Ultimately, it requires us to consider that all of our actions have outcomes and, in the case of medical intervention, we should be sure that we must always consider the harm that will be caused by this intervention.

Let us consider that there are two approaches that we could take in our pursuit of knowledge of learning and teaching: that of true scholarship of learning and teaching, and that of ignorance of new techniques of learning and teaching. (We’ll leave enthusiasm and ability to the side for the time being.) While this is falsely dichotomous, we can fix this by defining scholarship as starting at ‘knowing that other techniques exist and change might not kill you’, with everything else below that as ‘ignorance of new techniques’.

Now let us consider the impact of both of these bases, in terms of enthusiasm. If someone has any energy at all, then they will be able to apply techniques in the classroom. If they are more energetic then they will apply with more vigour and any effect will be amplified. If these are useful and evidentially supported techniques, then we would expect benefit. If these are folk pedagogies or traditions that have long been discredited then any vigour will be applied to an innately useless or destructive technique. In the case of an inert teacher, neither matters. It is obvious then that the minimum harm is to employ techniques that will reward vigour with sound outcomes: so we must either use validated techniques or explore new techniques that will work.

Now let us look at ability. If a teacher is ‘gifted’ (or profoundly experienced)  then he or she will be more likely to carry the class, pretty much regardless. However, what if a teacher is not so much of a star? Then, in this case, we start to become dependent once again upon the strength of the underlying technique or pedagogy. Otherwise, we risk harming our students by applying bad technique because of insufficient ability to correct it. Again, do no harm requires us to provide techniques that will survive the average or worse-than-average teacher, which requires a consideration of load, development level, reliance upon authority and so on – for student and teacher.

I believe that this argues that, yes, we are professionally bound to confirm our techniques and approaches and, if a better approach is available, evaluate it and adopt it. To do anything else risks doing harm and we cannot do this and remain professional. We are intervening with our students all the time – if we didn’t feel that our approach had worth or would change lives then we wouldn’t be doing it. If intervention and guidance are at our core then we must adopt something like the first, do no harm maxim because it gives us a clear signpost on decisions that could affect a student for life.

One of the greatest problems we face is potentially those people who are highly enthused and deeply undereducated in key areas of modern developments of teaching. As Kurt von Hammerstein-Equord would have said:

One must beware of anyone who is [undereducated] and [very enthusiastic] — [s/he] must not be entrusted with any responsibility because [s/he] will always cause only mischief.

If your best volunteer is also your worst nightmare, how do you resolve this when doing so requires you to say “This is right but you are wrong.” Can you do so without causing enormous problems that may swamp the benefit of doing so?

What about the legal issues? Do we risk heading into the murky world of compliance if we add a legal layer – will an ethical argument be enough?

What do you think about it?


The Philosophical Angle

Socrates drank hemlock after being found guilty of corrupting the minds of the youth of Athens, and impiety. Seneca submitted to the whims of Nero when the Emperor, inevitably, required that his old tutor die. Seneca’s stoicism was truly tested in this, given that he slashed his veins, took poison, jumped in a warm bath and finally had to be steamed to death before Nero’s edict that he kill himself was finally enacted. I, fortunately, expect no such demonstrations of stoic fortitude from my students but, if we are to think about their behaviour and development as self-regulating beings, then I think that a discussion of their personal philosophy becomes unavoidable. We have talked about the development state, their response to authority, their thoughts on their own thinking, but what of their philosophy?

If you are in a hurry and jump in your car, every red light between you and your destination risks becoming a personal affront, an enraging event that defies your expectation of an ‘all-green’ ride into town. There is no reason why you should expect such favours from the Universe, whatever your belief system, but the fact that this is infuriating to you remains. In the case of the unexpected traffic light, which sounds like the worst Sherlock Holmes story ever, the worst outcome is that you will be late, which may have a variety of repercussions. In preparing assignment work, however, a student may end up failing with far more dire and predictable results.

“Watson, I shall now relate the entire affair through Morse tapped pipe code and interpretative dance.”

While stoicism attracts criticism, understandably, because it doesn’t always consider the fundamentally human nature of humans, being prepared for the unforeseen is a vital part of any planning process. Self-regulation is not about drawing up a time table that allows you to fit in everything that you know about, it is about being able to handle your life and your work when things go wrong. Much as a car doesn’t need to be steered when it is going in a straight line and meeting our requirements, it is how we change direction when we know the road and when a kangaroo jumps out that are the true tests of our ability to manage our resources and ourselves.

Planning is not everything, as anyone who has read Helmuth von Moltke the Elder or von Clausewitz will know: “no plan survives contact with the enemy”. In this case, however, the enemy is not just those events that seek to confound us, it can be us as well! You can have the best plan in the world that relies upon you starting on Day X, and yet you don’t. You may have excellent reasons for this but, the fact remains, you have now introduced problems into your own process. You have met the enemy and it is you. This illustrates the critical importance of ensuring that we have an accurate assessment of our own philosophies – and we do have to be very honest.

There is no point in a student building an elaborate time management plan that relies upon them changing the habits of a lifetime in a week. But this puts the onus upon us as well: there is no point in us fabricating a set of expectations that a student cannot meet because they do not yet have a mature philosophy for understanding what is required. We don’t give up (of course!) but we must now think about how we can scaffold and encourage such change in a manageable way. I find reflection very handy, as I’ve said before, as watching students write things like “I planned for this but then I didn’t do it! WHY?” allows me to step in and discuss this at the point that the student realises that they have a problem.

I am not saying that a student who has a philosophy of “Maybe one day I will pass by accident” should be encouraged to maintain such lassitude, but we must be honest and realise that demanding that their timeliness and process maturity spring fully-formed from their foreheads is an act of conjuring reserved only for certain Greek Gods. (Even Caligula couldn’t manage it and he had far greater claim to this than most.) I like to think of this in terms of similarity of action. If anything I do is akin to walking up to someone and yelling “You should hand in on time, do better!” then I had better re-think my strategy.

The development of a personal philosophy, especially when you may not have ever been exposed to some of the great exemplars, is a fundamentally difficult task. You first need to understand that such a concept exists, then gain the vocabulary for discussing it, then interpret your current approach and see the value of change. Once you have performed all of those tasks, then we can start talking about getting from A to B. If you don’t know what I’m talking about or can’t understand why it’s important, or even discuss core concepts, then I’m yelling at you in the corridor and you’ll nod, compliantly, until I go away. Chances of you taking positive steps in the direction that I want? Very low. Probably, nil. And if it does happen, either it’s accidental or you didn’t actually need my help.

I try to be stoic but I must be honest and say that if Nero sentenced me to death, I’d nod, say “I expected that”, then put on some fast saxophone music and leg it up over the seven hills and far away. I don’t think I’d ever actually expect true stoicism from most of my students. but a simple incorporation of the fact that not everything works out as you think it will would be a definite improvement over the current everything will work out in my favour expectation that seems to be the hallmark of the more frequently disappointed and distressed among them. The trick is that I first have to make them realise that this is something that, with thought, they can not only fix but use to make a genuine, long-lasting and overwhelmingly positive change in their lives.


Surely, I can’t believe that I would have thought…

Anyone with students has become used to what I shall (extremely loosely) refer to as the argument of lazy denial, where a student uses one of the following in a sentence, when discussing a technical issue:

  • Surely…
  • I can’t believe…
  • I would have thought…

Now, used rhetorically, where you place a deliberately short-term doubt in someone’s mind and then follow it up with the facts, there is no real problem with most of these. My problem is when a student uses this in order to dismiss an idea, based on an isolated opinion or a very limited understanding of the issues. As I joked recently on someone’s Facebook, I’ve told my students that starting any technical discussion question with “Surely…” is an indication that further research has to take place.

Yes, yes, I’m making a point and enough of my students know about it to occasionally rib me with its deliberate usage but this just emphasises that they’re thinking about things. It’s very easy to infer a comfortable denial to a situation based on limited experience. This could be covered as being a hasty generalisation, jumping to conclusions, appeal to incredulity or wishful thinking, but it’s really an excuse to express disbelief without having to provide any evidence other than “Nahhhh.” And, ultimately, because very little work is being done here, I’m just going to call it lazy denial.

My intention is not, of course, to stop people speaking naturally but it’s to help my students think about framing an argument, which requires knowing enough about the area to be able to construct, and respond to, an argument. Research usually consists of knowing enough to know what you don’t know, which can usually be explained far more succinctly than saying “Surely, someone would have carried out action <x>”. There are legitimate ways to express this sentiment, after you’ve done the reading. “I’ve looked through all of the literature I can find and it appears that no-one appears to have tried <x>.”

(Regrettably, as in all things scientific, not finding something doesn’t prove its non-existence. As exhaustive literature searches are becoming harder and harder with the growth of the data corpus, we have to be very circumspect about how we make statements such as “no-one has done this” because it is more than a little embarrassing when someone stands up at the end of your talk and says “Urm, we did”.)

Once we’ve gone looking and discussed the area, we’re all looking at the same problem in the same way. Rather than making sweeping statements that are, to be honest, often a little condescending because you’re speaking as if your opinion is so blindingly obvious that it must have been tried, we can really appreciate the discovery of  a hole in the recorded knowledge: a place where we can make a contribution.

This is not to say that everything is this formal and there have been many fine semi-research discussions carried out that have used these terms but, when we’re sitting around trying to work towards a solution or my students are trying to work out their research direction, this starts to become important.

I suppose this reveals more about me than it does about my students…