Aside: My New York Story

I’m a story-teller. It infuses my work. I share the stories and realities of my life in order to explain points that I think other people could appreciate. Today, I’m telling you my New York story. It’s not rags-to-riches and I’m only triumphing over my own stupidity rather than terrible obstacles. I fight to give opportunity to others but my own tales are infused with my own levels of privilege. If that bothers you, probably best to stop reading now.

Like many New York stories, this one is all about how I never lived in New York. Losing both Bowie and Alan Rickman in a few days has made me think about that city again. This is the story of how I loved a city but we never ended up together.

Aerial perspective photo of a lot of tall buildings in midtown Manhattan, facing north to Central Park.

Oh, Oh! New York!

Even in the home counties towns in the UK, the adults spoke about New York as if it were Olympus, Shangri-La and a wild west town all rolled into one. You could be incredibly cool in Hampshire just by having been to America. If you had been to New York, and survived, you were some kind of god. They had different music, different cars, different money. In a Britain collapsing under the 1970s, America was a golden place.

I grew up in the 70s and was fortunate enough to hear Bowie hit the UK scene hard, to see early Doctor Who, to be mentally invigorated by very demanding progressive UK kids’ TV, to hear a Police album when they were just starting out and then, even more fortunately, we left the home counties and, because my Mum was amazingly brave and strong, we made it across the sea to Australia, where a better life awaited us.

I shrugged off Britain in weeks but New York never left me.

I continued to grow up in one of the Australian state capitals (Australian population is mostly concentrated into cities on the ocean) and it was nice, but it was no New York. I’m not sure I’ve told anyone this but, in my head, I was always going to America. That’s what success was defined as when I was a boy: you were a traveller, you did interesting things and that meant America. When every other Brit was going to the Costa del Sol and baking their skin, the interesting people were pale, thin and had walked around in magical places like Central Park, Times Square and along Broadway. They knew about music and understood what the Tarkus artwork meant on that ELP album. They even knew what “prog-rock” meant. They were art, life and wonder.

When I finished University, I started to think seriously about America. I had visited by then, and that is a story in itself, and finally seen New York. Mid-winter. Just after Christmas. Quiet and grey, on the cusp of 1995 and 1996. It did not quite amaze but it do not disappoint, even though I was walking around with a bung knee after slipping in unfamiliar snow. I began to think about how I could get there.

But one thing became clear as I thought about it. I didn’t know how to get to the New York that I wanted to be part of. I was creative but I wasn’t an amazing artist or musician and the New York I wanted to be part of was the bubbling, creative, amazing community of 1970s. I was a middling rhythm guitarist, a karaoke-tolerable singer, an abstract artist (I still can’t draw very well), an enthusiastic poet, but, mostly, I was a computer guy. I could go and get a job but all I would be doing would be living in (or near, most likely) New York and never becoming part of my vision of NYC. Tech support in Shangri-La was not what I wanted.

I had kept an idea of where I wanted to go in my head but I had never turned that into an intention to actually go there. A goal with no plans had turned into a life without much direction. (I was lucky enough to fall in love and start a real journey and adventure locally, just as I realised that I had never set my cap for New York, but that’s another story.)

There’s a poem I love, Ithaka by C. P. Cavafy, and it talks of setting out on a magnificent journey, full of adventures and monsters, in search of the island of Ithaka as part of a glorious ancient Greek adventure. But the most important part of the poem, for me, is the end:

Ithaka gave you the marvellous journey.
Without her you would not have set out.
She has nothing left to give you now.

And if you find her poor, Ithaka won’t have fooled you.
Wise as you will have become, so full of experience,
you will have understood by then what these Ithakas mean.

As Cavafy and all of the Greek authors of myth before him note, the journey is the thing. When you arrive at your goal, if you have thought about it for long enough, then you may find it was not as good as you thought it would be. Memory may have tricked you, things may have changed. But if it caused you to start a wonderful journey, then it was worthwhile. No, not just worthwhile, it was magical. It was transcendence and enlightenment.

But, by not seeing New York as a goal and leaving it as a dream, I never set myself upon the journey and thus I deprived myself of both the adventure and the chance of achieving my dream one day.

Let’s be realistic, I was never going to live in the 1970s New York that I idolised, unless I found a time machine, but if I had actually made some career and life choices that would have seen me head to America in the 90s, I still would have made it there. It would have seen me experiencing a New York that, while not the one I thought of, would have been a capstone to an amazing journey. But, because I didn’t align myself to realise my New York dream, it didn’t happen.

My long-time love affair with New York was a fantasy and we’re both lucky that we never moved it beyond that point. I would have ended up being bitter and resentful, New York didn’t need another tech support person pretending to be an artist. You need more than attraction and the frisson of distance to have a relationship.

This year, I have reassessed my goals and dreams. I am deciding which of these will define my journey and give my life structure for the next decade or two. Where can I find new wisdom? Where can I find the experiences that will take me to new and amazing places, physically or mentally? I have been successful in a number of things but I really need to focus on the goal to make sure that I get the most out of the journey.

I’m not the same person I was back in the 90s. A lot of thinking has happened, a lot of growing has happened, a lot of love has happened. I’m more comfortable with the softer definition of myself as a communicator, an educator, an artist and even a philosopher. This small journey was triggered by the realisation that I had never chosen what to do or where to go. There’s a natural pause at this stage and it’s time to set a new heading. Where do I go from here?

The point of my New York Story is a simple one: assuming that nothing else gets in the way, you’re unlikely to get somewhere unless you actually set out for it. We often mistake what we’re doing with what we want to do, the necessary aims of our work with our real goals in life. We do something today because we did it yesterday and that means we’ll do it again tomorrow. Perhaps we should only do it tomorrow if it’s the best thing to do.

There are many things in my life that I don’t want (and don’t need) to change. But I look at Cavafy’s poem and I can smell the sea winds, hear the sails fill, and the helm is asking me where to go next.

 


Teaching for (current) Humans

da Vinci's Vitrvuian Man. Human figure with arms and legs outstretched showing the ratios of the perfect form.

Leonardo’s experiments in human-octopus engineering never received appropriate recognition.

I was recently at a conference-like event where someone stood up and talked about video lectures. And these lectures were about 40 minutes long.

Over several million viewing sessions, EdX have clearly shown that watchable video length tops out at just over 6 minutes. And that’s the same for certificate-earning students and the people who have enrolled for fun. At 9 minutes, students are watching for fewer than 6 minutes. At the 40 minute mark, it’s 3-4 minutes.

I raised this point to the speaker because I like the idea that, if we do on-line it should be good on-line, and I got a response that was basically “Yes, I know that but I think the students should be watching these anyway.” Um. Six minutes is the limit but, hey, students, sit there for this time anyway.

We have never been able to unobtrusively measure certain student activities as well as we can today. I admit that it’s hard to measure actual attention by looking at video activity time but it’s also hard to measure activity by watching students in a lecture theatre. When we add clickers to measure lecture activity, we change the activity and, unsurprisingly, clicker-based assessment of lecture attentiveness gives us different numbers to observation of note-taking. We can monitor video activity by watching what the student actually does and pausing/stopping a video is a very clear signal of “I’m done”. The fact that students are less likely to watch as far on longer videos is a pretty interesting one because it implies that students will hold on for a while if the end is in sight.

In a lecture, we think students fade after about 15-20 minutes but, because of physical implications, peer pressure, politeness and inertia, we don’t know how many students have silently switched off before that because very few will just get up and leave. That 6 minute figure may be the true measure of how long a human will remain engaged in this kind of task when there is no active component and we are asking them to process or retain complex cognitive content. (Speculation, here, as I’m still reading into one of these areas but you see where I’m going.) We know that cognitive load is a complicated thing and that identifying subgoals of learning makes a difference in cognitive load (Morrison, Margulieux, Guzdial)  but, in so many cases, this isn’t what is happening in those long videos, they’re just someone talking with loose scaffolding. Having designed courses with short videos I can tell you that it forces you, as the designer and teacher, to focus on exactly what you want to say and it really helps in making your points, clearly. Implicit sub-goal labelling, anyone? (I can hear Briana and Mark warming up their keyboards!)

If you want to make your videos 40 minutes long, I can’t stop you. But I can tell you that everything I know tells me that you have set your materials up for another hominid species because you’re not providing something that’s likely to be effective for current humans.

 


At least they’re being honest

I was inspired to write this by a comment about using late penalties but dealing slightly differently with students when they owned up to being late. I have used late penalties extensively (it’s school policy) and so I have a lot of experience with the many ways students try to get around them.

6874balance_scale

Like everyone, I have had students who have tried to use honesty where every other possible way of getting the assignment in on time (starting early, working on it before the day before, miraculous good luck) has failed. Sometimes students are puzzled that “Oh, I was doing another assignment from another lecturer” isn’t a good enough excuse. (Genuine reasons for interrupted work, medical or compassionate, are different and I’m talking about the ambit extension or ‘dog ate my homework’ level of bargaining.)

My reasoning is simple. In education, owning up to something that you did knowing that it would have punitive consequences of some sort should not immediately cause things to become magically better. Plea bargaining (and this is an interesting article of why that’s not a good idea anywhere) is you agreeing to your guilt in order to reduce your sentence. But this is, once again, horse-trading knowledge on the market. Suddenly, we don’t just have a temporal currency, we have a conformal currency, where getting a better deal involves finding the ‘kindest judge’ among the group who will give you the ‘lightest sentence’. Students optimise their behaviour to what works or, if they’re lucky, they have a behaviour set that’s enough to get them to a degree without changing much. The second group aren’t mostly who we’re talking about and I don’t want to encourage the first group to become bargain-hunting mark-hagglers.

I believe that ‘finding Mr Nice Lecturer’ behaviour is why some students feel free to tell me that they thought someone else’s course was more important than mine, because I’m a pretty nice person and have a good rapport with my students, and many of my colleagues can be seen (fairly or not) as less approachable or less open.

We are not doing ourselves or our students any favours. At the very least, we risk accusations of unfairness if we extend benefits to one group who are bold enough to speak to us (and we know that impostor syndrome and lack of confidence are rife in under-represented groups). At worst, we turn our students into cynical mark shoppers, looking for the easiest touch and planning their work strategy based on what they think they can get away with instead of focusing back on the learning. The message is important and the message must be clearly communicated so that students try to do the work for when it’s required. (And I note that this may or may not coincide with any deadlines.)

We wouldn’t give credit to someone who wrote ‘True’ and then said ‘Oh, but I really meant False’. The work is important or it is not. The deadline is important or it is not. Consequences, in a learning sense, do not have to mean punishments and we do not need to construct a Star Chamber in our offices.

Yes, I do feel strongly about this. I completely understand why people do this and I have also done this before. But after thinking about it at length, I changed my practice so that being honest about something that shouldn’t have happened was appreciated but it didn’t change what occurred unless there was a specific procedural difference in handling. I am not a judge. I am not a jury. I want to change the system so that not only do I not have to be but I’m not tempted to be.


A good goal

I set out to make this year an amazing year. A beautiful year for education. But it’s already challenging in some ways and I knew it was going to be.

cropped-dscn00371.jpg

You know what it feels like, sometimes.

What a great time to find this quote from President Obama. It’s one that I think everyone in education should try to remember and apply to what we’re doing because every time we have new students in our classes, we will never again get the chance to start well with them.

“I want us to be able when we walk out this door to say we couldn’t think of anything else that we didn’t try to do—that we didn’t shy away from a challenge because it was hard. That we weren’t timid, or got tired, or somehow were thinking about the next thing because there is no next thing. This is it. Never in our lives again will we have the chance to do as much good as we do right now. I want to make sure that we maximize it.”

State of the Union Address, President Barack Obama, 2016.


A quick note on direction

Wind vane

I’m getting some great comments, on and off the blog, about possible solutions to the problems I’m putting up, as well as thoughts on some of my examples.

Firstly, thank you, everyone! Secondly, I am deliberately starting slowly and building up, to reframe all of these arguments in terms of aesthetics, fitness for purpose and clarity. (Beauty, goodness and truth, again.) I am not trying to make anything appear worse than it is but I’m teasing out some points to show why we should be seeking to change practice that is both widespread and ingrained.

I will make a quick note that Raymond Lister raised about my thought experiment with the two students who split the knowledge, in that I don’t differentiate between skills and knowledge (true) and I am talking about an educational design where no work has been done to identify which areas have to be mastered in order to progress (also true). This is totally deliberate on my part, because it reflects a lot of current practice, not because I think it’s what we should be doing. I will be returning to, and extending this, example over time.

(Raymond does great work in a lot of areas dear to my heart and we will be returning to some of his work in our peregrinations, especially the SOLO taxonomy and Bloom’s mappings. Until then, here is his Google Scholar link for you to read some very interesting papers. And I could not agree more that there is no programming gene!)


The Illusion of a Number

Rabbit? Duck? Paging Wittgenstein!

I hope you’ve had a chance to read William Rapaport’s paper, which I referred to yesterday. He proposed a great, simple alternative to traditional grading that reduces confusion about what is signalled by ‘grade-type’ feedback, as well as making things easier for students and teachers. Being me, after saying how much I liked it, I then finished by saying “… but I think that there are problems.” His approach was that we could break all grading down into: did nothing, wrong answer, some way to go, pretty much there. And that, I think, is much better than a lot of the nonsense that we pretend we hand out as marks. But, yes, I have some problems.

I note that Rapaport’s exceedingly clear and honest account of what he is doing includes this statement. “Still, there are some subjective calls to make, and you might very well disagree with the way that I have made them.” Therefore, I have license to accept the value of the overall scholarship and the frame of the approach, without having to accept all of the implementation details given in the paper.  Onwards!

I think my biggest concern with the approach given is not in how it works for individual assessment elements. In that area, I think it shines, as it makes clear what has been achieved. A marker can quickly place the work into one of four boxes if there are clear guidelines as to what has to be achieved, without having to worry about one or two percentage points here or there. Because the grade bands are so distinct, as Rapaport notes, it is very hard for the student to make the ‘I only need one more point argument’ that is so clearly indicative as a focus on the grade rather than the learning. (I note that such emphasis is often what we have trained students for, there is no pejorative intention here.) I agree this is consistent and fair, and time-saving (after Walvoord and Anderson), and it avoids curve grading, which I loathe with a passion.

However, my problems start when we are combining a number of these triaged grades into a cumulative mark for an assignment or for a final letter grade, showing progress in the course. Sections 4.3 and 4.4 of the paper detail the implementation of assignments that have triage graded sub-tasks. Now, instead of receiving a “some way to go” for an assignment, we can start getting different scores for sub-tasks. Let’s look at an example from the paper, note 12, to describe programming projects in CS.

  • Problem definition 0,1,2,3
  • Top-down design 0,1,2,3
  • Documented code
    • Code 0,1,2,3
    • Documentation 0,1,2,3
  • Annotated output
    • Output 0,1,2,3
    • Annotations 0,1,2,3

Total possible points = 18

Remember my hypothetical situation from yesterday? I provided an example of two students who managed to score enough marks to pass by knowing the complement of each other’s course knowledge.  Looking at the above example, it appears (although not easily) to be possible for this situation to occur and both students to receive a 9/18, yet for different aspects. But I have some more pressing questions:

  1. Should it be possible for a student to receive full marks for output, if there is no definition, design or code presented?
  2. Can a student receive full marks for everything else if they have no design?

The first question indicates what we already know about task dependencies: if we want to build them into numerical grading, we have to be pedantically specific and provide rules on top of the aggregation mathematics. But, more subtly, by aggregating these measures, we no longer have an ‘accurately triaged’ grade to indicate if the assignment as a whole is acceptable or not. An assignment with no definition, design or code can hardly be considered to be a valid submission, yet good output, documentation and annotation (with no code) will not give us the right result!

The second question is more for those of us who teach programming and it’s a question we all should ask. If a student can get a decent grade for an assignment without submitting a design, then what message are we sending? We are, implicitly, saying that although we talk a lot about design, it’s not something you have to do in order to be successful. Rapaport does go on to talk about weightings and how we can emphasis these issues but we are still faced with an ugly reality that, unless we weight our key aspects to be 50-60% of the final aggregate, students will be able to side-step them and still perform to a passing standard. Every assignment should be doing something useful, modelling the correct approaches, demonstrating correct techniques. How do we capture that?

Now, let me step back and say that I have no problem with identifying the sub-tasks and clearly indicating the level of performance using triage grading, but I disagree with using it for marks. For feedback it is absolutely invaluable: triage grading on sub-tasks will immediately tell you where the majority of students are having trouble, quickly. That then lets you know an area that is more challenging than you thought or one that your students were not prepared for, for some reason. (If every student in the class is struggling with something, the problem is more likely to lie with the teacher.) However, I see three major problems with sub-task aggregation and, thus, with final grade aggregation from assignments.

The first problem is that I think this is the wrong kind of scale to try and aggregate in this way. As Rapaport notes, agreement on clear, linear intervals in grading is never going to be achieved and is, very likely, not even possible. Recall that there are four fundamental types of scale: nominal, ordinal, interval and ratio. The scales in use for triage grading are not interval scales (the intervals aren’t predictable or equidistant) and thus we cannot expect to average them and get sensible results. What we have here are, to my eye, ordinal scales, with no objective distance but a clear ranking of best to worst. The clearest indicator of this is the construction of a B grade for final grading, where no such concept exists in the triage marks for assessing assignment quality. We have created a “some way to go but sometimes nearly perfect” that shouldn’t really exist. Think of it like runners: you win one race and you come third in another. You never actually came second in any race so averaging it makes no sense.

The second problem is that aggregation masks the beauty of triage in terms of identifying if a task has been performed to the pre-determined level. In an ideal world, every area of knowledge that a student is exposed to should be an important contributor to their learning journey. We may have multiple assignments in one area but our assessment mechanism should provide clear opportunities to demonstrate that knowledge. Thus, their achievement of sufficient assignment work to demonstrate their competency in every relevant area of knowledge should be a necessary condition for graduating. When we take triage grading back to an assignment level, we can then look at our assignments grouped by knowledge area and quickly see if a student has some way to go or has achieved the goal. This is not anywhere near as clear when we start aggregating the marks because of the mathematical issues already raised.

Finally, the reduction of triage to mathematical approximation reduces the ability to specify which areas of an assessment are really valuable and, while weighting is a reasonable approximation to this, it is very hard to use a mathematical formula with more and more ‘fudge factors’, a term Rapaport uses, to make up for the fact that this is just a little too fragile.

To summarise, I really like the thrust of this paper. I think what is proposed is far better, even with all of the problems raised above, at giving a reasonable, fair and predictable grade to students. But I think that the clash with existing grading traditions and the implicit requirement to turn everything back into one number is causing problems that have to be addressed. These problems mean that this solution is not, yet, beautiful. But let’s see where we can go.

Tomorrow, I’ll suggest an even more cut-down version of grading and then work on an even trickier problem: late penalties and how they affect grades.


Assessment is (often) neither good nor true.

If you’ve been reading my blog over the past years, you’ll know that I have a lot of time for thinking about assessment systems that encourage and develop students, with an emphasis on intrinsic motivation. I’m strongly influenced by the work of Alfie Kohn, unsurprisingly given I’ve already shown my hand on Focault! But there are many other writers who are… reassessing assessment: why we do it, why we think we are doing it, how we do it, what actually happens and what we achieve.

Screen Shot 2016-01-09 at 6.50.12 PM

In my framing, I want assessment to be as all other aspects of education: aesthetically satisfying, leading to good outcomes and being clear and what it is and what it is not. Beautiful. Good. True. There are some better and worse assessment approaches out there and there are many papers discussing this.  One of these that I have found really useful is Rapaport’s paper on a simplified assessment process for consistent, fair and efficient grading. Although I disagree with some aspects, I consider it to be both good, as it is designed to clearly address a certain problem to achieve good outcomes, and it is true, because it is very honest about providing guidance to the student as to how well they have met the challenge. It is also highly illustrative and honest in representing the struggle of the author in dealing with the collision of novel and traditional assessment systems. However, further discussion of Rapaport is for the near future. Let me start by demonstrating how broken things often are in assessment, by taking you through a hypothetical situation.

Thought Experiment 1

Two students, A and B, are taking the same course. There are a number of assignments in the course and two exams. A and B, by sheer luck, end up doing no overlapping work. They complete different assignments to each other, half each and achieve the same (cumulative bare pass overall) marks. They then manage to score bare pass marks in both exams, but one answers only the even questions and only answers the odd. (And, yes, there are an even number of questions.) Because of the way the assessment was constructed, they have managed to avoid any common answers in the same area of course knowledge. Yet, both end up scoring 50%, a passing grade in the Australian system.

Which of these students has the correct half of the knowledge?

I had planned to build up to Rapaport but, if you’re reading the blog comments, he’s already been mentioned so I’ll summarise his 2011 paper before I get to my main point. In 2011, William J. Rapaport, SUNY Buffalo, published a paper entitled “A Triage Theory of Grading: The Good, The Bad and the Middling.” in Teaching Philosophy. This paper summarised a number of thoughtful and important authors, among them Perry, Wolff, and Kohn. Rapaport starts by asking why we grade, moving through Wolff’s taxonomic classification of assessment into criticism, evaluation, and ranking. Students are trained, by our world and our education systems to treat grades as a measure of progress and, in many ways, a proxy for knowledge. But this brings us into conflict with Perry’s developmental stages, where students start with a deep need for authority and the safety of a single right answer. It is only when students are capable of understanding that there are, in many cases, multiple right answers that we can expect them to understand that grades can have multiple meanings. As Rapaport notes, grades are inherently dual: a representative symbol attached to a quality measure and then, in his words, “ethical and aesthetic values are attached” (emphasis mine.) In other words, a B is a measure of progress (not quite there) that also has a value of being … second-tier if an A is our measure of excellence. A is not A, as it must be contextualised. Sorry, Ayn.

When we start to examine why we are grading, Kohn tells us that the carrot and stick is never as effective as the motivation that someone has intrinsically. So we look to Wolff: are we critiquing for feedback, are we evaluating learning, or are we providing handy value measures for sorting our product for some consumer or market? Returning to my thought experiment above, we cannot provide feedback on assignments that students don’t do, our evaluation of learning says that both students are acceptable for complementary knowledge, and our students cannot be discerned from their graded rank, despite the fact that they have nothing in common!

Yes, it’s an artificial example but, without attention to the design of our courses and in particular the design of our assessment, it is entirely possible to achieve this result to some degree. This is where I wish to refer to Rapaport as an example of thoughtful design, with a clear assessment goal in mind. To step away from measures that provide an (effectively) arbitrary distinction, Rapaport proposes a tiered system for grading that simplifies the overall system with an emphasis on identifying whether a piece of assessment work is demonstrating clear knowledge, a partial solution, an incorrect solution or no work at all.

This, for me, is an example of assessment that is pretty close to true. The difference between a 74 and a 75 is, in most cases, not very defensible (after Haladyna) unless you are applying some kind of ‘quality gate’ that really reduces a percentile scale to, at most, 13 different outcomes. Rapaport’s argument is that we can reduce this further and this will reduce grade clawing, identify clear levels of achieve and reduce marking load on the assessor. That last point is important. A system that buries the marker under load is not sustainable. It cannot be beautiful.

There are issues in taking this approach and turning it back into the grades that our institutions generally require. Rapaport is very open about the difficulties that he has turning his triage system into an acceptable letter grade and it’s worth reading the paper to see that discussion alone, because it quite clearly shows what

Rapaport’s scheme clearly defines which of Wolff’s criteria he wishes his assessment to achieve. The scheme, for individual assessments, is no good for ranking (although we can fashion a ranking from it) but it is good to identify weak areas of knowledge (as transmitted or received) for evaluation of progress and also for providing elementary critique. It says what it is and it pretty much does it. It sets out to achieve a clear goal.

The paper ends with a summary of the key points of Haladyna’s 1999 book “A Complete Guide to Student Grading”, which brings all of this together.

Haladyna says that “Before we assign a grade to any students, we need:

  1. an idea about what a grade means,
  2. an understanding of the purposes of grading,
  3. a set of personal beliefs and proven principles that we will use in teaching

    and grading,

  4. a set of criteria on which the grade is based, and, finally,
  5. a grading method,which is a set of procedures that we consistently follow

    in arriving at each student’s grade. (Haladyna 1999: ix)

There is no doubt that Rapaport’s scheme meets all of these criteria and, yet, for me, we have not yet gone far enough in search of the most beautiful, most good and most true extent that we can take this idea. Is point 3, which could be summarised as aesthetics not enough for me? Apparently not.

Tomorrow I will return to Rapaport to discuss those aspects I disagree with and, later on, discuss both an even more trimmed-down model and some more controversial aspects.


Beauty Attack I: Assessment

For the next week, I’m going to be applying an aesthetic lens to assessment and, because I’m in Computer Science, I’ll be focusing on the assessment of Computer Science knowledge and practice.

How do we know if our students know something? In reality, the best way is to turn them loose, come back in 25 years and ask the people in their lives, their clients, their beneficiaries and (of course) their victims, the same question: “Did the student demonstrate knowledge of area X?”

This is not available to us as an option because my Dean, if not my Head of School, would probably peer at me curiously if I were to suggest that all measurement of my efficacy be moved a generation from now. Thus, I am forced to retreat to the conventions and traditions of assessment: it is now up to the student to demonstrate to me, within a fixed timeframe, that he or she has taken a firm grip of the knowledge.

We know that students who are prepared to learn and who are motivated to learn will probably learn, often regardless of what we do. We don’t have to read Vallerand et al to be convinced that self-motivated students will perform, as we can see it every day. (But it is an enjoyable paper to read!) Yet we measure these students in the same assessment frames as students who do not have the same advantages and, thus, may not yet have the luxury or capacity of self-motivation: students from disadvantaged backgrounds, students who are first-in-family and students who wouldn’t know auto-didacticism if it were to dance in front of them.

How, then, do we fairly determine what it means to pass, what it means to fail and, even more subtly, what it means to pass or fail well? I hesitate to invoke Foucault, especially when I speak of “Discipline and Punish” in an educational setting, but he is unavoidable when we gaze upon a system that is dedicated to awarding ranks, graduated in terms of punishment and reward. It is strange, really, that were many patients to die under the hand of a surgeon for a simple surgery, we would ask for an inquest, but many students failing under the same professor in a first-year course is merely an indicator of “bad students”. So many of our mechanisms tell us that students are failing but often too late to be helpful and not in a way that encourages improvement. This is punishment. And it is not good enough.

A picture of Michel Foucault, mostly head shot, with his hand on his forehead, wearing glasses, apparently deep in contemplation.

Foucault: thinking about something very complicated, apparently.

Our assessment mechanisms are not beautiful. They are barely functional. They exist to provide a rough measure to separate pass from fail, with a variety of other distinctions that owe more to previous experience and privilege in many cases than any higher pedagogical approach.

Over the next week, I shall conduct an attack upon the assessment mechanisms that are currently used in my field, including my own, in the hope of arriving at a mechanism of design, practice and validation that is pedagogically pleasing (the aesthetic argument again) and will lead to outcomes that are both good and true.


Getting it wrong

It’s fine to write all sorts of wonderful statements about theory and design and we can achieve a lot in thinking about such things. But, let’s be honest, we face massive challenges in the 21st Century and improved thinking and practice in education is one of the most important contributions we can make to future generations. Thus, if we want to change the world based upon our thinking, then all of our discussions have no use if we can’t develop something that’s going to achieve our goals. Dewey’s work provide an experimental, even instrumental, approach to the American philosophical school of pragmatism. To briefly explain this term in the specific meaning, I turn to William James, American psychologist and philosopher.

Pragmatism asks its usual question. “Grant an idea or belief to be true,” it says, “what concrete difference will its being true make in anyone’s actual life? How will the truth be realized? What experiences will be different from those which would obtain if the belief were false? What, in short, is the truth’s cash-value in experiential terms?”
William James, Pragmatism (1907)

(James is far too complex to summarise with one paragraph and I am using only one of his ideas to illustrate my point. Even James’ scholars disagree on how to interpret many of his writings. It’s worth reading him and Hegel at the same time as they square off across the ring quite well.)

Portrait, side view, of a man before middle-age, with dark receding hair, wearing a waistcoat and red tie over a white shirt. He appears to be painting.

Portrait of William James by John La Farge, circa 1859

What will be different? How will we recognise or measure it? What do we gain by knowing if we are right or wrong? This is why all good education researchers depend so heavily on testing their hypotheses in the space where they will make an impact and there is usually an obligation to look at how things are working before and after any intervention. This places further obligation upon us to evaluate what has occurred and then, if our goals haven’t been achieved, change our approach further. It’s a simple breakdown of roles but I often think as educational work in three heavily overlapping areas: practice, scholarship and research. Practice should be applying techniques that achieve our goals, scholarship involves the investigation, dissemination and comparison of these techniques, and research builds on scholarship to evaluate practice in ways that will validate and develop new techniques – or invalidate formerly accepted ones as knowledge improves. This leads me to my point: evaluating your own efforts to work out how to do better next time.

There are designers, architects, makers and engineers who are committed to the practice of impact design, where (and this is one definition):

“Impact design is rooted in the core belief that design can be used to create positive social, environmental and economic change, and focuses on actively measuring impact to inform and direct the design process.”  Impact Design Hub, About.

Thus, evaluation of what works is essential for these practitioners. The same website recently shared some designers talking about things that went wrong and what they learned from the process.

If you read that link, you’ll see all sorts of lessons: don’t hand innovative control to someone who’s scared of risk, don’t ignore your community, don’t apply your cultural values to others unless you really know what you’re doing, and don’t forget the importance of communication.

Writing some pretty words every day is not going to achieve my goal and I need to be reminded of the risks that I face in trying to achieve something large – one of which is not actually working towards my own goals in a useful manner! One of the biggest risks is confusing writing a blog with actual work, unless I use this medium to do something. Over the coming weeks, I hope to show you what I am doing as I move towards my very ambitious goal of “beautiful education”. I hope you find the linked article as useful as I did.

 


Dewey’s Pedagogic Creed

As I’ve noted, the space I’m in is not new, although some of the places I hope to go with it are, and we have records of approaches to education that I think fit well into an aesthetic framing.

As a reminder, I’m moving beyond ‘sensually pleasing’ in the usual sense and extending this to the wider definition of aesthetics: characteristics that define an approach or movement. However, we can still see a Cubist working as both traditionally aesthetically pleasing and also beautiful because of its adherence to the Cubist aesthetic. To draw on this, where many art viewers find a large distance between them and an art work, it is often attributable to a conflict over how beauty is defined in this context. As Hegel noted, beauty is not objective, it is our perspective and our understanding of its effect upon us (after Kant) that contributes greatly to the experience.

A black and white chest and head portrait of John C. Dewey, an older man with centre-parted white hair, a trimmed mostly dark haired moustache and oval wire-framed glasses.

John C. Dewey. Psychologist, philosopher, educator, activist and social critic. Also, inspiration.

Dewey’s Pedagogic Creed was published in 1897 and he sought to share his beliefs on what education was, what schools were, what he considered the essential subject-matter of education, the methods employed, and the essential role of the school in social progress. I use the word ‘beliefs’ deliberately as this is what Dewey published: line after line of “I believe…” (As a note, this is what a creed is, or should be, as a set of beliefs or aims to guide action. The word ‘creed’ comes to us from the Latin credo, which means “I believe”.) Dewey is not, for the most part, making a religious statement in his Creed although his personal faith is expressed in a single line at the end.

To my reading, and you know that I seek characteristics that I can use to form some sort of object to guide me in defining beautiful education, many of Dewey’s points easily transfer to characteristics of beauty. For example, here are three lines from the work:

  1. I believe that education thus conceived marks the most perfect and intimate union of science and art conceivable in human experience.
  2. I believe that with the growth of psychological science, giving added insight into individual structure and laws of growth; and with growth of social science, adding to our knowledge of the right organization of individuals, all scientific resources can be utilized for the purposes of education.
  3. I believe that under existing conditions far too much of the stimulus and control proceeds from the teacher, because of neglect of the idea of the school as a form of social life.

Dewey was very open about what he thought the role of school was, he saw it as the “fundamental method of social progress and reform“. I believe that he saw education, when carried out correctly, as being a thing that was beautiful, good and true and his displeasure with what he encountered in the schools and colleges of the late 19th/early 20th Century is manifest in his writings. He writes in reaction to an ugly, unfair, industrialised and mechanistic system and he wants something that conforms to his aesthetics. From the three lines above, he seeks education that is grounded in the arts and science, he wants to use technology in a positive way and he wants schools to be a vibrant and social community.

And this is exactly what the evidence tells us works. The fact that Dewey arrived at this through a focus on equity, opportunity, his work in psychology and his own observations is a testament to his vision. Dewey was rebelling against the things he could see were making children hate education.

I believe that next to deadness and dullness, formalism and routine, our education is threatened with no greater evil than sentimentalism.

John Dewey, School Journal vol. 54 (January 1897), pp. 77-80

Here, sentimentalism is where we try to evoke emotions without associating them with an appropriate action: Dewey seeks authenticity and a genuine expression. But look at the rest of that list: dead, dull, formal and routine. Dewey would go on to talk about schools as if they were prisons and over a hundred years later, we continue to line students up into ranks and bore them.

I have a lot of work to do as I study Dewey and his writings again with my aesthetic lens in place but, while I do so, it might be worth reading the creed. Some things are dated. Some ideas have been improved upon with more research, including his own and we will return to these issues. But I find it hard to argue with this:

I believe that the community’s duty to education is, therefore, its paramount moral duty. By law and punishment, by social agitation and discussion, society can regulate and form itself in a more or less haphazard and chance way. But through education society can formulate its own purposes, can organize its own means and resources, and thus shape itself with definiteness and economy in the direction in which it wishes to move.

ibid.