Last Lecture Blues

Posted: June 7, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | Leave a comment

I delivered the last lecture, technically the review and revision lecture, for my first year course today. As usual, when I’ve had a good group of students or a course I enjoy, the relief in having reduced my workload is minor compared to the realisation that another semester has come to an end and that this particular party is over.

Hellooooo???

Today’s lecture was optional but we still managed to get roughly 30% of the active class along. Questions were asked, questions were discussed, outline answers were given and then, although they all say and listened until I’d finished a few minutes late, they were all up and gone. The next time I’ll see most of them is at the exam, a few weeks from now. After that? It depends on what I teach. Some of these students I’ll run into over the years and we’ll actually get to know each other. Some may end up as my honours or post-graduate students. Some will walk out of the gates this semester and never return.

Now, hear me out, because I’m not complaining about it, but this is not the easiest job in the world. Done properly, education requires constant questioning, study, planning, implementing, listening, talking and, above all, dealing with the fact that you may see the best student you ever have for a maximum of 6 months. It is, however, a job that I love, a job that I have a passion for and, of course, in many ways it’s a calling more than a job.

One of the things I’ve had a chance to reflect on in this blog is how much I enjoy my job, while at the same time recognising how hard it is to do it well. Many times, the students I need to speak to most are those who contact me least, who up and fade away one day, leaving me wondering what happened to them.

At the end of the semester, it’s a good time to ask myself some core questions and see if I can give some good answers:

  1. Did I do the best job that I could do, given the resources (structures, curriculum, computers etc) that I had to work with?
  2. Did I actively seek out the students who needed help, rather than just waiting for people to contact me?
  3. Did I look for pitfalls before I ran into them?
  4. Did I look after the staff who were working with or for me, and mentor them correctly?
  5. Did I try to make everything that I worked with an awesome experience for my students?

This has been the busiest six months of my life and one of my joys has been walking into a lecture theatre, having written the course, knowing the material and losing myself in an hour of interactive, collaborative and enjoyable knowledge exchange with my students. Despite that, being so busy, sometimes I didn’t quite have the foresight that I should had had and my radar range was measured in days rather than weeks. Don’t get me wrong, everything got done, but I could have tried to locate troubled students more actively, and some minor pitfalls nearly got me.

However, I think that we still delivered a great course and I’m happy with 1, 4 and 5. I aimed for awesome and I think we hit it fairly often. 2 and 3 needed work but I’ve already started making the required changes to make this better.

On reflection, I’d give myself an 8ish/10 but, of course, that’s not enough. Overall, in the course, because of the excellent support from my co-lecturer and my teaching staff, the course itself I’d push up into the 9-pluses. I, however, should be up there as well and right now, I’m too busy.

So, it’s time for some rebalancing into the new semester. Some more structure for identifying problems students. Looking at things a little earlier. And aiming for an awesome 10/10 for my own performance next semester.

To all my students, past and present, it’s been fantastic. Best of luck with your exams!

The Shortest Distance Between Two Points Is The Internet

Posted: June 5, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | 2 Comments

MY CHEESE!

I’ve been noticing a slight upwards trend in readership, as I’ve previously noted. On Friday, however, the readership exploded. 209 views on Friday, 259 views on Saturday. (Normally, I’m lucky to get a third of that.) One person, who obviously needs some help, viewed 99 things in one hour. (Even I’ve never done that.)

It was pretty obvious that something had happened but the other shoe didn’t drop until today, when a comment confirmation request came in. I’d been mentioned in a radio show and podcast for electronics enthusiasts and professionals – The Amp Hour!

Dave Jones had found my post on the bra ad that managed to combine sexism and insulting engineers and had, in his own words, been dismayed by it. His comment, and a link on the website, and suddenly I have some (quite possibly heavily confused) people reading my blog and wondering what they’d wandered into.

I saw the comment confirmation request, followed it back, commented, Dave responded, loop closed.

Except, of course, for how amazingly cool it is that something I wrote about an issue that bothered me found someone else, who it also bothered, who mentioned it, which sent people to me, which sent me to him, and then we basically met up in the corridor, nodded at each other and said “Hi”.

Of course, this would be far more impressive if Dave was in Antarctica, but he’s actually in Sydney. He went looking for stuff about the ad because he saw it and it bugged him – and he found my blog.

But this is still pretty cool. We’re a thousand kilometres apart (or so) and we’ve both agreed that the ad is rubbish. Twenty years ago, we would have had to have met or one of us would have had to write an article that got published on paper and distributed to me. I certainly wouldn’t know about Dave’s expertise in Internet Dating (seriously?) or passion for geocaching.

This is the world that that I have to prepare my students for. A world where their comments, good or bad, can travel for thousand of kilometres. A world where presence doesn’t have to mean physical presence. Of course, as educators, that applies to our classrooms and spaces as well. Our world does not have to be bricks and mortar, bench seats and blackboards.

We’re trying to make connections: knowledge, ideas, people, the future. We already appear to have the infrastructure to do it – do we have all of the right tools and the drive and vision to realise it?

I have no idea but I hope that we do. In the meantime, I’ll be over here, looking at the walls and thinking about the shortest distance between two points.

Eating Your Own Dog Food (How Can I Get Better at Words with Friends?)

Posted: June 4, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , | 1 Comment

I am currently being simultaneously beaten in four games of Words with Friends. This amuses me far more than it bugs me because it appears that, despite having a large vocabulary, a (I’m told) quick wit and being relatively skilled in the right word in the right place – I am rather bad at a game that should reward at least some of these skills.

One of the things that I dislike, and I know that my students dislike, is when someone stands up and says “To solve problem X, you need to take set of actions S.” Then, when you come to X, or you find that person’s version of a solution to X, it’s not actually S that is used. It’s “S-like” or “S-lite” or “Z, which looks like an S backwards and sounds like it if you’re an American with a lisp.”

There’s a term I love called “eating your own dog food” (Wikipedia link) that means that a company uses the products that it creates in order to solve the problems for which a customer would buy their products. It’s a fairly simple mantra: if you’re making the best thing to solve Problem X, then you should be using it yourself when you run across Problem X. Now,of course, a company can do this by banning or proscribing any other products but this misses the point. At it’s heart, dogfooding means that, in a situation where you are free to choose, you make a product so good that you would choose it anyway.

It speaks to authenticity when you talk about your product and it provides both goals and thinking framework. The same thing works for education – if I tell someone to take a certain approach to solve a problem, then it should be one that I would use as well.

So, if a student said to me “I am bad at this type of problem,” I’d start talking to them to find out exactly what they’re good and bad at, get them to analyse their own process, get them to identify some improvement strategies (with my guidance and suggestions) and then put something together to get it going. Then we’d follow up, discuss what happened, and (with some careful scaffolding) we’d iteratively improve this as far as we could. I’d also be open to the student working out whether the problem is actually one that they need to solve – although it’s a given that I’ll have a strong opinion if it’s something important.

So, let me eat my own dog food for this post, to help me get better at Words with Friends, to again expose my thinking processes but also to demonstrate the efficacy of doing this!

Step 1: What’s the problem?

So, I can get reasonable scores at Words with Friends but I don’t seem to be winning. Words with Friends is a game that rewards you for playing words with “high value” tiles on key positions that add score multipliers. The words QATS can be worth 13 or 99 depending on where it is placed. You have 7 randomly selected tiles with different letters, and a range of values for letters in a 1:1 association, but must follow strict placement and connection rules. In summary, a Words with Friends game is a connected set of tiles, where each set of tiles placed must form a valid word once set placement is complete, and points are calculated from the composition and placement of the tiles, but bonus spaces on the board only count once. The random allocation of letters means that you have to have a set of strategies to minimise the negative impact of a bad draw and to maximise the benefit of a good draw. So you need a way of determining the possible moves and then picking the best one.

Some simple guidelines that help you to choose words can be formed along the lines of the number of base points by letter (so words featuring Q, X or J will be worth more because these are high value letters), the values of words will tend to increase as the word length increases as there are more letters with values to count (although certain high value letters cannot be juxtaposed – QXJJXWY is not a word, sadly), but both of these metrics are overshadowed by the strategic placement of letters to either extend existing words (allowing you to recount existing tiles and extending point 2) or to access the bonus spaces. Given that QATS can be worth 99 points as a four-letter word if played in the right place, it might be worth ignoring QUEUES earlier if think you can reach that spot.

Step 2: So where is my problem?

After thinking about my game, I realised that I wasn’t playing Words with Friends properly, because I wasn’t giving enough thought to the adversarial nature of the occupancy of the bonus spaces. My original game was more along the lines of “look at letters, look at board, find a good word, play it.” As a result, any occupancy of the bonus spaces was a nice-to-have, rather than a must-have. I also didn’t target placement that allowed me to count tiles already on the board and, looking at other games, my game is a loose grid compared to the tight mesh that can earn very large points.

I’m also wasn’t thinking about the problem space correctly. There are a fixed number of tiles in the game, with known distribution. As tiles are played, I know how many tiles are left and that up to 14 of them are in my and my opponent’s hand. If I know how many tiles there are of each letter, I can play with a reasonable idea of the likelihood of my opponent’s best move. Early on, this is hard, but that’s ok, because we can both play in a way that doesn’t give a bonus tile advantage. Later on, it’s probably more useful.

Finally, I was trying to use words that I knew, rather than words that are legal in Words with Friends. I had no idea that the following were acceptable until (at least once out of desperation) I tried them. Here are some you might (nor might now) know: AA, QAT, ZEE, ZAS, SCARP, DYNE. The last one is interesting, because it’s a unit of force, but BRIX, a unit used to measure concentration (often of sugar) isn’t a legal word.

So, I had three problems, most of which relate to the fact that I’m more used to playing “Take 2” (a game played with Scrabble tiles but no bonus spaces) than “Scrabble” itself, where the bonus spaces are crucial.

Step 3: What are the strategies for improvements?

The first, and most obvious, strategy is to get used to playing in the adversarial space and pay much closer attention to which bonus spaces I leave open in my play and to increase my recounting of existing tiles. The second is to start keeping track of tiles that are out and play to the more likely outcome. Finally, I need to get a list of which words are legal in Words with Friends and, basically, learn them.

Step 4: Early outcomes

After getting thrashed in my first games, I started applying the first strategy. I have since achieved words worth over 100 points and, despite not winning, the gap is diminishing. So this appears to be working.

The second and the third… look, it’s going to sound funny but this seems like a lot of work for a game. I quite like playing the best word I can think of without having to constrain myself to play some word I’m never going to actually use (when we’re up to our elbows in aa, I will accept your criticism then) or sit there eliminating tiles one-by-one (or using an assistant to do it). Given that I’m not even sure that this is the way people actually play, I’m probably better off playing a lot of games and naturally picking up words that occur, rather than trying to learn them all in one go.

Of course, if a student said something along the last lines to me, then they’re saying that they don’t mind not succeeding. In this case, it’s perfectly true. I enjoy playing and, right now, I don’t need to win to enjoy the game.

Just as well really, I think I’m about to lose four games within a minute of each other. That’s four in a row – pity, if there were three of them I could do a syzygy joke.

Step 5: Discussion and Iteration

So, here’s the discussion and my chance to think about whether my strategies need modification to achieve my original goal. Now, if I keep that goal at winning, then I do need to keep iterating but I have noticed that with a simple change of aiming more a the bonuses, I get a good “Yeah” from a high points word that probably won’t be matched by winning a game.

To wrap up, having looked at the problem, thought through it and make some constructive suggestions regarding improvement, I’ve not only improved my game but I’ve improved my understanding and enjoyment of the activity. I feel far more in control of my hideous performance and can now talk to more people about other ways to improve that maintain that enjoyment.

Now, of course, I imagine that a million WwF players are going to jump in and say “nooooo! here’s how you do it.” Please do so! Right now I’m talking to myself but I’d love some guidance for iterative improvement.

Codes of Conduct: Being a Grown-Up.

Posted: May 31, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | Leave a comment

I always hope that my students are functioning at a higher level, heading towards functional adulthood, to some extent. After all, if they need to go to the bathroom, they can usually manage that in a clean and tidy manner. They dress themselves. They can answer questions. So why do some of them act like children when it comes to good/bad behaviour?

I searched for “adult child” and found this. I think Craig Ferguson should sue.

I was reading Darlena’s blog post about one of Rafe Esquith’s books and she referred to Rafe’s referral to Kohlberg’s Six Levels of Moral Development, which I ‘quote-quote’ here:

  1. I do not want to get into trouble.
  2. I want a reward.
  3. I want to please someone.
  4. I always follow the rules.
  5. I am considerate of other people.
  6. I have a personal code of behaviour.

I’ve been talking around these points for a while, in terms of the Perry classifications of duality, multiplicity and commitment. What disappoints me the most is when I have to deal with students who are either trying not to get into trouble or only work for reward – and these are their prime motivations. There’s a world of difference between having students who do things because they have worked through everything we’ve talked about and decided to commit to that approach (step 6 in this scale) and those who only do it because they feel that they will get punished if they don’t.

I always say that I expect a lot of my students and, fairly early on, I do expect them to have formed a personal code of conduct. Yes, I expect them to be timely in their submissions, but because they understand that assignment placement is deliberate and assists them in knowledge formation. Yes, I expect them to not plagiarise or cheat, but because to do so deprives them of learning opportunities. I expect them not to talk in class because they don’t want to deprive other people of learning opportunities (which is a bit of points 5 and 6).

I press this point a lot. I say that I reward what they know, as long as it’s relevant, rather than punishing them for getting things wrong. I encourage them to participate, to be aware of other people, to interact and work with me to make the knowledge transfer more effective – to allow them to construct the mental frameworks required to produce the knowledge for themselves.

I really don’t think it’s good enough to say “Well, students always do X and what can you do?” I have a number of people in my classes who have discovered, to their mounting amazement, that I basically won’t accept behaviour that doesn’t meet reasonable standards. I mean what I say when I say things and I don’t change my mind just because someone asks me. I’m tough on plagiarism and cheating. I don’t let people bully me or other people. And, amazingly, I don’t see many of these behaviours in my class.

I encourage a constructive and positive approach for all of my students – but the basis of this is that they have to establish a personal code of conduct that I can work with. If they go down this path, then everything else tends to follow and we can go a fantastic educational journey together. If they’re still stuck, doing the minimum they can get away with, because they don’t want to get yelled at, then my first (and far more difficult) task is to reach them, try and get them to think beyond using this as their only motivator.

Now, of course, the golden rule is that if you want a student to do something, then giving marks for it is the best way to go – and that’s a technique I use, and I’ve discussed it before. But it’s never JUST the marks. There’s always  reward in terms of scaffolding, or personal satisfaction, or insight. I want fiero! I also don’t want the students to do things just because I ask them to, because they want to please me. I have a middling amount of lecturing charisma but I’m always aware that I have to be content first/showmanship second. If I do that, then students are less likely to fall into the trap of trying to do things just because I ask them to.

I’m really not the kind of teacher who needs an apple on the desk. (I already have two iMacs and a MacBook Air. Ba-dum-*ting*)

Number 4 is one that I really want to steer people away from. Yes, rules should be followed – except where they shouldn’t. You may not know this but it is completely legitimate for a solider in the Australian Army to refuse to follow an illegal order. (Yes, it will probably not go very well but it’s still an option.) If a soldier, who is normally bound by the chain of command to follow orders, believes the order to be illegal (“No prisoners” being one of them) they don’t have to follow it. Australian soldiers are encouraged to exercise discretion and thought because that makes them better soldiers – they can fill in the blanks when the situation changes and potentially improve things. The price, of course, is that a thinker thinks.

Same for students. I want students who change the world, who make things better, who may occasionally walk on the grass to get to that bright new future even when the signs say ‘stay off the grass’. However, without a personal code of conduct, which rules you can bend or break are going to be fairly arbitrarily selected and are far more likely to have a selfish focus. We want rule bending in the face of sound ethics, not rationalisation.

As I said, it’s a lot to ask of students but, as I’ve always said, if I don’t ask for it, and tell people what I want, I can’t expect it and I certainly can’t build on it.

Rush, Rush: Baby, Please Plan To Submit Your Work Earlier Than The Last Minute

Posted: May 29, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , | 4 Comments

Sorry, Paula Abdul, but I had to steal a song lyric from you.

AND MANGLE IT!

This is the one of the first pictures that comes up when you search for ‘angry Paula Abdul”. Sorry, Lamar.

I’ve been marking the first “process awareness” written report from my first-year students. A one-page PDF that shows their reflections on their timeliness and assignment performance to date and how they think that they can improve it or maintain it. There have been lots of interesting results from this. From about 100 students, I’ve seen many reports along the lines of “I planned, I assigned time, SO WHY DIDN’T I FOLLOW THE PLAN?” or “Wow, I never realised how much I needed a design until I was stuck in the middle of a four-deep connection of dynamic arrays.”

This is great – understanding why you are succeeding or failing allows you to keep doing the things that work, and change the things that don’t. Before this first-year curriculum restructure, and this course, software development process awareness could avoid our students until late second- or third-year. Not any more. You got run over by the infamous Library prac? You know, you should have written a design first. And now my students have all come to this realisation as well. Two of my favourite quotes so far are:

“[Programming in C++] isn’t hard but it’s tricky.”

and

“It’s not until you have a full design [that you can] see the real scope of the project.”

But you know I’m all about measurement so, after I’d marked everything, I went back and looked at the scores, and the running averages. Now here’s the thing. The assignment was marked out of 10. Up until 2 hours before the due date, the overall average was about 8.3. For the last two hours, the average dropped to 7.2. The people commenting in the last two hours were making loose statements about handing up late, and not prioritising properly, but giving me enough that I could give them some marks. (It’s not worth a lot of marks but I do give marks for style and reflection, to encourage the activity.) The average mark is about 8/10 usually. So, having analysed this, I gave the students some general feedback, in addition to the personalised feedback I put on every assignment, and then told them about that divide.

The fact that the people before the last minute had the marks above the average, and that the people at the last minute had the marks below.

One of the great things about a reflection assignment like this is that I know that people are thinking about the specific problem because I’ve asked them to think about it and rewarded them with marks to do so. So when I give them feedback in this context and say “Look – planned hand-in gets better marks on average than last-minute panic” there is a chance that this will get incorporated into the analysis and development of a better process, especially if I give firm guidelines on how to do this in general and personalised feedback. Contextualisation, scaffolding… all that good stuff.

There are, as always, no guarantees, but moving this awareness and learning point forward is something I’ve been working on for some time. In the next 10 days, the students have to write a follow-up report, detailing how they used the lessons they learnt, and the strategies that they discussed, to achieve better or more consistent results for the next three practicals. Having given them guidance and framing, I now get to see what they managed to apply. There’s a bit of a marking burden with this one, especially as the follow-up report is 4-5 pages long, but it’s worth it in terms of the exposure I get to the raw student thinking process.

Apart from anything else, let me point out that by assigning 2/10 for style, I appear to get reports at a level of quality where I rarely have to take marks away and they are almost all clear and easy to read, as well as spell-checked and grammatically correct. This is all good preparation and, I hope, a good foundation for their studies ahead.

What Did You Learn From Higher Education?

Posted: May 27, 2012 | Author: | Filed under: Education | Tags: , , , , , , , | 1 Comment

Today has been games day at my house. We’ve played some Arkham Horror and Lords of Waterdeep, one collaborative and one highly competitive board game, and it has been a lot of fun. It’s been the standard group of Australians around the table. Eight people, six with PhDs, and two currently studying for them. (That is, of course, not a serious comment on standard. A lot of my friends, including my wife, are University trained and have post-graduate qualifications.)

I was wondering what to talk about today and, at breakfast this morning, my wife suggested that I ask our guests what they got from going to University – what they learnt? (We gave them lunch and dinner so it wasn’t too much of an imposition. 🙂 )

My wife’s answer to the question was that she learnt to keep going, keep putting effort into her work to get something good out of a course. My answer was that I learnt that it was never over, even at the end of a degree – that you could always do something new, something different, change career. (Yes, we’re similar but not quite the same.)

When I asked my friends, I got a variety of responses, because we’d been playing games for over 8 hours and we’d had wine with dinner. One said that he was still learning, but that he thought it was more about the process than the output. One said that she learnt how to drink and keep up with men (I suspect this wasn’t her most serious answer). Another said that, although it sounded cynical, he thought it was often better to be convincing than right. One, who I work with closely, said that truly horrific educators cannot spoil kids if the kids are really keen. One said that he learnt programming.

The last answer got laughs from around the table, as did many of the answers – as did the question. There are always going to be a range of answers to a question like this: a person’s reaction to this question, especially when I told them was going to publish it, is generally going to be framed self-consciously. However, all of them are using the skills that they learnt in Higher Education and all of them at least started PhD studies, even if they hadn’t completed them yet. There is no doubt, in this group, that the University if a useful thing, even if particular instances are not fantastic exemplars of that.

But it’s an interesting question. What did you learn from your foray into higher ed, if you’ve done it. What do you think of when you think of higher education? If you’re going there, what are you expecting to learn? If you’ve never had any direct exposure, what do you think that people learn when they’re there?

The Confusing Message: Sourcing Student Feedback

Posted: May 26, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , | Leave a comment

Once, for a course which we shall label ‘an introduction to X and Y’, I saw some feedback from a student that went as follows. A single student, on the same feedback form, and in adjacent text boxes, gave these answers:

What do you like most about this course: the X

What would you like to see happen to improve the course: less X, more Y!

Now, of course, this not inherently contradictory but, honestly, it’s really hard to get the message here. You think that X is great but less useful than Y, although you like X more? You’re a secret masochist and you like to remove pleasure from your life?

As (almost) always, the problem here is that we these two questions, asked in adjacent text boxes, are asking completely different things. Survey construction is an art, a dark and mysterious art, and a well-constructed survey will probably not answer a question once, in one way. It will ask the same question in multiple ways, sometimes in the negative, to see if the “X” and “not ( not (X))” scores line up for each area of interest. This, of course, assumes that you have people who are willing to fill out long surveys and give you reliable answers. This is a big assumption. Most of the surveys that I work with have to fit into short time frames and are Likert-based with text boxes. Not quite yes/no tick/flick but not much more and very little opportunity for mutually interacting questions.

The Red Tick of Courage!

Our student experience surveys are about 10 questions long with two text boxes and are about the length that we can fit into the end of a lecture and have the majority of students fill out and return. From experience, if I construct larger surveys, or have special ‘survey-only’ sessions, I get poor participation. (Hey, I might just be doing it wrong. Tips and help in the comments, please!)

Of course, being Mr Measurement, I often measure things as side effects of the main activity. Today, I held a quiz in class and while everyone was writing away, I was actually getting a count of attendees because they were about to hand up cards for marking. This gives me an indicator of attendance and, as it happens, two weeks away from the end of the course, we’re still getting good attendance. (So, I’m happy.) I can also see how the students are doing with fundamental concepts so I can monitor that too.

I’m fascinated by what students think about their experience but I need to know what they need based on their performance, so that I can improve their performance without having to work out what they mean. The original example would give me no real insight into what to do and how to improve – so I can’t really do anything with any certainty. If the student had said “I love X but I feel that we spent too much time on it and it could be just as good with a little less.” then I know what I can do.

I also sometimes just ask for direct feedback in assignments, or in class, because then I’ll get the things that are really bugging or exciting people. That also gives me the ability to adapt to what I hear and ask more directed questions.

Student opinion and feedback can be a vital indicator of our teaching efficacy, assuming that we can find out what people think rather than just getting some short and glib answers to questions that don’t really probe in the right ways, where we never get a real indication of their thoughts. To do this requires us to form a relationship, to monitor, to show the value of feedback and to listen. Sadly, that takes a lot more work than throwing out a standard form once a semester, so it’s not surprising that it’s occasionally overlooked.

Proscription and Prescription: Bitter Medicine for Teachers

Posted: May 24, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , | Leave a comment

Australia is a big country. A very big country. Despite being the size of the continental USA, it has only 22,000,000 people, scattered across the country and concentrated in large cities. This allows for a great deal of regional variation in terms of local culture, accents (yes, there is more than one Australian accent) and local industry requirements. Because of this, despite having national educational standards and shared ideas of what constitutes acceptable entry levels for University, there are understandable regional differences in the primary, secondary and tertiary studies.

Maintaining standards is hard, especially when you start to consider regional issues – whose standards are you maintaining. How do you set these standards? Are they prescriptions (a list of things that you must do) or proscriptions (a list of things that you mustn’t do)? There’s a big difference in course and program definition depending upon how you do this. If you prescribe a set textbook then everyone has to use it to teach with but can bring in other materials. If you proscribe unauthorised textbooks then you have suddenly reduced the amount of initiative and independence that can be displayed by your staff.

Excuse me, Doctor, you appear to be writing in invisible ink…

As always, I’m going to draw an analogue with our students to think about how we guide them. Do we tell them what we want and identify those aspects that we want them to use, or do we tell them what not to do, limit their options and then look surprised when they don’t explore the space and hand in something that conforms in a dull and lifeless manner?

I’m a big fan of combining prescription, in terms of desirable characteristics, and proscription, in terms of pitfalls and traps, but in an oversight model that presents the desirable aspects first and monitors the situation to see if behaviour is straying towards the proscribed. Having said that, the frequent flyers of the proscription world, plagiarism and cheating, always get mentioned up front – but as the weak twin of the appropriate techniques of independent research, thoughtful summarisation, correct attribution and doing your own work. Rather than just saying “DO NOT CHEAT”, I try to frame it in terms of what the correct behaviour is and how we classify it if someone goes off that path.

However, any compulsory inclusions or unarguable exclusions must be justified for the situation at hand – and should be both defensible and absolutely necessary. When we start looking at a higher level, above the individual school to the district, to the region, to the state, to the country, any complex set of prescriptions and proscriptions is very likely to start causing regional problems. Why? Because not all regions are the same. Because not all districts have the money to meet your prescriptions. Because not all cultures may agree with your proscriptions.

This post was triggered by a post from a great teacher I know, to whom I am also related, who talked about having to take everything unofficial out of her class. Her frustration with this, the way it made her feel, the way it would restrict her – an award winning teacher – made me realise how privileged I am to work in a place where nobody really ever tells me what to do or how to teach. While it’s good for me to remember that I am privileged in this regard, perhaps it’s also good to think about the constant clash between state, bureaucracy and education that exist in some other places.

If We’re Going To Measure, Let’s Measure Properly: Teaching Isn’t a NASCAR Race

Posted: May 22, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , | 1 Comment

I’ve been reading a Huff Post piece on teacher assessment, entitled “Carolyn Abbott, The Worst 8th Grade Math Teacher In New York City, Victim Of Her Own Success”, where a teacher, Carolyn Abbot, at a gifted and talented school in Manhattan was rated being the worst teacher in 8th grade.

The problem, it appears, is the measurement used where your contribution is based upon whether your students have performed better or worse than last year on the Teacher Data Report, a measure used to assess contribution to English and Math. So here’s the problem. The teacher taught maths to grades 7 and 8 and her Grade 7 students achieved at the 98th percentile for their test in 2009. Therefore, according to the Teacher Data Reports modelling process, the same students should have achieved 97th percentile in their Grade 8 tests the following year. They only managed 89th percentile. Abbot had made a significant negative contribution to her students, by this logic, and her ranking was the lowest in NYC 8th grade mathematics teachers.

Yes, you read that right. She’s kept the students in the 1.5-2 standard deviations above the norm category. The students have moved up a year and are now starting to run into the puberty zone, always fun, they’re still scoring in the 89th percentile – and she’s the worst teacher in NYC. Her students struggle with the standardised testing itself: the non-mathematical nature of the tests, the requirement to put in a single answer when the real answer is potentially more complex, the fact that multiple choice can be trained for (rather than test anything) – and they’re still kicking out at the +85 level. Yet, she’s the worst 8th grade math teacher in NYC.

This also goes against one of my general principles of assessment, in that the performance of someone else affects the assessment of your performance. (Yes, that leaves me at odds with national testing schemes, because I don’t see a way that they can be meaningfully calibrated across many different teaching systems and economic influences. It’s obvious that New York haven’t worked it out properly for one system and one economy!) Having a notion of acceptable and unacceptable is useful here. Having a notion of exemplary, acceptable and unacceptable is useful here. Having a notion of best and worst is meaningless, because all these teachers could score 100/100 and one scores 100/99 and they’re the worst. Ranking must be combined with standards of acceptability where professional practice is required. This isn’t a NASCAR race: in teaching, everyone can cross the line in a way that they win.

I am a big fan of useful, carefully constructed and correctly used measurement but this story is an example of what happens if you come up with a simple measure that gives you a single number that isn’t much use but is used as if it means something. Now, if over time, you saw a large slide in scores from one teacher and that dropped down low enough, then maybe this number would mean something but any time that you simple number has to come with an explanation – it’s not that simple anymore.

In this case, what’s worse is that the rankings were published with names. Names of teachers and names of schools. Abbott’s boss reassured her that he would still put her up for tenure but felt he had to warn her that someone else might take these rankings into account.

Abbott’s ranking doesn’t matter to her much anymore, because this teacher has now left teaching and is undertaking a PhD in Mathematics instead. Great for us at University because having good teachers who then successfully complete PhDs often works out very well – they’re highly desirable employees in many ways. Not so good for the students at her school who have been deprived of a teacher who managed to get a group of kids to the 98th percentile on Grade 7 Math, providing a foundation that will probably be with them for their whole lives (even if we quibble and it’s down to the 89th) and giving them a better start for their academic future.

But that’s ok, kids, because she was the worst teacher you’d ever have. Oh, of course, there’s another new “worst” teacher because that’s how our ranking system works. Sorry about that. Good luck, Carolyn Abbott!

How Do We Equate Investment in On-Line Learning Versus Face-to-Face?

Posted: May 21, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , | Leave a comment

It’s relatively early/late where I am because my jet lag has hit and I’m not quite sure which zone I’m in. This is probably a little rambly but I hope that there’s at least something to drive some thinking or discussion. No doubt, there is an excellent link that I can’t find, which describes all of this, so please send it to me or drop it into the comments!

One of the things I’m thinking about at the moment is how to quantify an equivalent ‘Teaching Dollar’ that reflects what an institution has to spend to support a student. Now, while this used to be based on investment in a bricks-and-mortar campus, a set number of hours and instructional support, we now have an extra dimension as blended learning spreads – the costs required for on-line and mobile components. We can look at a course in terms of course content, number of units contributing to degree progress, hopefully tie this into a quantification of requisite skill and knowledge components and then get a very rough idea of what we’re spending per “knowledge progress unit” in board dollar terms.

What’s the electronic equivalent of a lecture theatre? If I can run an entire course on-line, for n students, should I get the equivalent dollar support value as if they were on campus, using the expensive bricks-and-mortar facilities? If we spend $x on lecture theatres, what do we have to spend on on-line resources to have the equivalent utility in terms of teaching support? In a flipped classroom, where almost all informational review takes place outside of class and class is reserved for face-to-face activities, we can increase the amount of material because it’s far more efficient for someone to read something than it is for us to chalk-and-talk them through it. Do we measure the time that we spent producing electronic materials in terms of how much classroom time we could have saved or do we just draw a line under it and say “Well, this is what we should have been doing?”

On-line initiatives don’t work unless they have enough support, instructor presence, follow-up and quality materials. There are implicit production costs, distribution issues – it all has to be built on a stable platform. To do this properly takes money and time, even if it’s sound investment that constructs a solid base for future expansion. Physical lecture theatres are, while extremely useful, expensive to build, slow to build and they can only be used by one class at a time. We’re still going to allocate an instructor to a course but, as we frequently discuss, some electronic aspects require a producer as well to allow the lecturer to stay focused on teaching (or facilitating learning) while the producer handles some of the other support aspects. Where does the money for that producer come from? The money you save by making better use of your physical resources or the money you save by being able to avoid building a new lecture theatre at all. If we can quantity an equivalent dollar value, somehow, then we can provide a sound case for the right levels of support for on-line and the investment in mobile and, rather than look like we’re spending more money, we may be able to work out if we’re spending more money or whether it’s just in a different form.

I’m mulling on a multi-axis model that identifies the investment in terms of physical infrastructure, production support and electronic resources. With the spread of PDF slides and lecture recordings, we’re already pushing along the electronic resources line but most of this is low investment in production support (and very few people get money for a slide producer). The physical infrastructure investment and use is high right now but it’s shifting. So what I’m looking to do is get some points in three-space that represent several courses and see if I can define a plane (or surface) that represents equivalence along the axes. I have a nasty suspicion that not all of the axes are even on the same scales – at least one of them feels log-based to me.

If we define a point in the space (x,y,z) as (Physical, Production, Electronic) then, assuming that a value of 10 corresponds to “everything in a lecture theatre” for physical (and the equivalent of total commitment on the other axes), then the “new traditional” lecture of physical lectures with slide support and recordings would be (10,0,2) and a high quality on-line course would be (0,10,10). Below is a very sketchy graph that conveys almost no information on this.

Simple 3-Axis model in the most unclear graph ever.

It would be really easy if x+y+z = V, where V was always the same magnitude, but the bounding equation is, on for a maximum value of M in each axis, actually x+y+z <= 3M. Could we even have a concept of negative contribution, to reflect that lack of use of a given facility will have an impact on learning and teaching investment in the rest of our course, or in terms of general awareness?

I suspect that I need a lot more thought and a lot more sleep on this. Come back in a few months. 🙂

← Older Entries
Newer Entries →