Equity is the principal educational aesthetic

Posted: January 26, 2016 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , | Leave a comment

I’ve laid out some honest and effective approaches to the evaluation of student work that avoid late penalties and still provide high levels of feedback, genuine motivation and a scalable structure.

Scheme2

But these approaches have to fit into the realities of time that we have in our courses. This brings me to the discussion of mastery learning (Bloom). An early commenter noted how much my approach was heading towards mastery goals, where we use personalised feedback and targeted intervention to ensure that students have successfully mastered certain tiers of knowledge, before we move on to those that depend upon them.

A simple concept: pre-requisites must be mastered before moving on. It’s what much of our degree structure is based upon and is what determines the flow of students through courses, leading towards graduation. One passes 101 in order to go on to courses that assume such knowledge.

Within an individual course, we quickly realise that too many mastery goals starts to leave us in a precarious position. As I noted from my earlier posts, having enough time to do your job as designer or evaluator requires you to plan what you’re doing and keep careful track of your commitments. The issue that arises with mastery goals is that, if a student can’t demonstrate mastery, we offer remedial work and re-training with an eye to another opportunity to demonstrate that knowledge.

This can immediately lead to a backlog of work that must be completed prior to the student being considered to have mastered an area, and thus being ready to move on. If student A has completed three mastery goals while B is struggling with the first, where do we pitch our teaching materials, in anything approximating a common class activity, to ensure that everyone is receiving both what they need and what they are prepared for? (Bergmann and Sams’ Flipped Mastery is one such approach, where flipping and time-shifting are placed in a mastery focus – in their book “Flip Your Classroom”)

But even if we can handle a multi-speed environment (and we have to be careful because we know that streaming is a self-fulfilling prophecy) how do we handle the situation where a student has barely completed any mastery goals and the end of semester is approaching?

Mastery learning is a sound approach. It’s both ethically and philosophically pitched to prevent the easy out for a teacher of saying “oh, I’m going to fit the students I have to an ideal normal curve” or, worse, “these are just bad students”. A mastery learning approach tends to produces good results, although it can be labour intensive as we’ve noted. To me, Bloom’s approach is embodying one of my critical principles in teaching: because of the variable level of student preparation, prior experience and unrelated level of privilege, we have to adjust our effort and approach to ensure that all students can be brought to the same level wherever possible.

Equity is one of my principle educational aesthetics and I hope it’s one of yours. But now we have to mutter to ourselves that we have to think about limiting how many mastery goals there are because of administrative constraints. We cannot load up some poor student who is already struggling and pretend that we are doing anything other than delaying their ultimate failure to complete.

At the same time, we would be on shaky ground to construct a course where we could turn around at week 3 of 12 and say “You haven’t completed enough mastery goals and, because of the structure, this means that you have already failed. Stop trying.”

The core of a mastery-based approach is the ability to receive feedback, assimilate it, change your approach and then be reassessed. But, if this is to be honest, this dependency upon achievement of pre-requisites should have a near guarantee of good preparation for all courses that come afterwards. I believe that we can all name pre-requisite and dependency patterns where this is not true, whether it is courses where the pre-requisite course is never really used or dependencies where you really needed to have achieved a good pass in the pre-req to advance.

Competency-based approaches focus on competency and part of this is the ability to use the skill or knowledge where it is required, whether today or tomorrow. Many of our current approaches to knowledge and skill are very short-term-focussed, encouraging cramming or cheating in order to tick a box and move on. Mastering a skill for a week is not the intent but, unless we keep requiring students to know or use that information, that’s the message we send. This is honesty: you must master this because we’re going to keep using it and build on it! But trying to combine mastery and grades raises unnecessary tension, to the student’s detriment.

As Bloom notes:

Mastery and recognition of mastery under the present relative grading system is unattainable for the majority of students – but this is the result of the way in which we have “rigged” the educational system.

Bloom, Learning for Mastery, UCLA CSEIP Evaluation Comment, 1, 2, 1968.

Mastery learning is part and parcel of any competency based approach but, without being honest about the time constraints that are warping it, even this good approach is diminished.

The upshot of this is that any beautiful model of education adhering to the equity aesthetic has to think in a frame that is longer than a semester and in a context greater than any one course. We often talk about doing this but detailed alignment frequently escapes us, unless it is to put up our University-required ‘graduate attributes’ to tell the world how good our product will be.

We have to accept that part of our job is asking a student to do something and then acknowledging that they have done it, while continuing to build systems where what they have done is useful, provides a foundation to further learning and, in key cases, is something that they could do again in the future to the approximate level of achievement.

We have to, again, ask not only why we grade but also why we grade in such strangely synchronous containers. Why is it that a degree for almost any subject is three to five years long? How is that, despite there being nearly thirty years between the computing knowledge in the degree that I did and the one that I teach, they are still the same length? How are we able to have such similarity when we know how much knowledge is changing?

A better model of education is not one that starts from the assumption of the structures that we have. We know a lot of things that work. Why are we constraining them so heavily?

Can we do this? We already have.

Posted: January 15, 2016 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

How does one actually turn everything I’ve been saying into a course that can be taught? We already have examples of this working, whether in the performance/competency based models found in medical schools around the world or whether in mastery learning based approaches where do not measure anything except whether a student has demonstrated sufficient knowledge or skill to show an appropriate level of mastery.

An absence of grades, or student control over their grades, is not as uncommon as many people think. MIT in the United States give students their entire first semester with no grades more specific than pass or fail. This is a deliberate decision to ease the transition of students who have gone from being leaders at their own schools to the compressed scale of MIT. Why compressed? If we were to assess all school students then we would need a scale that could measure all levels of ability, from ‘not making any progress at school’ to ‘transcendent’. The tertiary entry band is somewhere between ‘passing school studies’ to ‘transcendent’ and, depending upon the college that you enter, can shift higher and higher as your target institution becomes more exclusive. If you look at the MIT entry requirements, they are a little coy for ‘per student’ adjustments, but when the 75th percentile for the SAT components is 800, 790, 790, and 800,800,800 would be perfect, we can see that any arguments on how demotivating simple pass/fail grades must be for excellent students have not just withered, they have caught fire and the ash has blown away. When the target is MIT, it appears the freshmen get their head around a system that is even simpler than Rapaport’s.

MIT_Dome_night1_Edit

Pictured: A highly prestigious University with some of the most stringent entry requirements in the world, which uses no grades in first semester.

Other universities, such as Brown, deliberately allow students to choose how their marks are presented, as they wish to deemphasise the numbers in order to focus on education. It is not a cakewalk to get into Brown, as these figures attest, and yet Brown have made a clear statement that they have changed their grading system in order to change student behaviour – and the world is just going to have to deal with that. It doesn’t seem to be hurting their graduates, from quotes on the website such as “Our 85% admission rate to medical school and 89% admission rate to law school are both far above the national average.”

And, returning to medical schools themselves, my own University runs a medical program where the usual guidelines for grading do not hold. The medical school is running on a performance/competency scheme, where students who wish to practise medicine must demonstrate that they are knowledgable, skilful and safe to practice. Medical schools have identified the core problem in my thought experiment where two students could have the opposite set of knowledge or skills and they have come to the same logical conclusion: decide what is important and set up a scheme that works for it.

When I was a solider, I was responsible for much of the Officer Training in my home state for the Reserve. We had any number of things to report on for our candidates, across knowledge and skills, but one of them was “Demonstrate the qualities of an officer” and this single item could fail an otherwise suitable candidate. If a candidate could not be trusted to one day be in command of troops on the battlefield, based on problems we saw in peacetime, then they would be counselled to see if it could be addressed and, if not, let go. (I can assure you that this was not used often and it required a large number of observations and discussion before we would pull that handle. The power of such a thing forced us to be responsible.)

We know that limited scale, mastery-based approaches are not just working in the vocational sector but in allied sectors (such as the military), in the Ivy league (Brown) and in highly prestigious non-Ivy league institutions such as MIT. But we also know of examples such as Harvey Mudd, who proudly state that only seven students since 1955 have earned a 4.0 GPA and have a post on the career blog devoted to “explaining why your GPA is so low” And, be in no doubt, Harvey Mudd is an excellent school, especially for my discipline. I’m not criticising their program, I’ve only heard great things about them, but when you have to put up a page like that? You’re admitting that there’s a problem but you are pushing it on to the student to fix it. But contrast that with Brown, who say to employers “look at our students, not their grades” (at least on the website).

Feedback to the students on their progress is essential. Being able to see what your students are up to is essential for the teacher. Being able to see what your staff and schools are doing is important for the University. Employers want to know who to hire. Which of these is the most important?

The students. It has to be the students. Doesn’t it? (Arguments for the existence of Universities as a self-sustaining bureaucracy system in the comments, if you think that’s a thing you want to do.)

This is not an easy problem but, as we can see, we have pieces of the solution all over the place. Tomorrow, I’m going to put in a place a cornerstone of beautiful assessment that I haven’t seen provided elsewhere or explained in this way. (Then all of you can tell me which papers I should have read to get it from, I can publish the citation, and we can all go forward.)

 

How Do We Recognise Mastery? What Is My Masterpiece?

Posted: July 8, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | 1 Comment

An artwork entitled “Masterpiece”. Click through for the webpage.

A few posts ago, and my goodness that’s a lot of words, I posted on issues of identity and examined the PhD in the light of it being a journeyman qualification, one that indicates the end of an apprenticeship and a readiness to go out into the world. That, however, is only half of the overall story of the apprentice, because there is a level above journeyman and that is, in all of its gendered glory, “master”. In the world of the trade and craft guilds, the designation of Mastery was only given when a journeyman applied to the guild and provided a piece of work that demonstrated their mastery of the appropriate craft. These works, if accepted, paved the way for journeyman to become Master, to become capable of training more apprentices and retaining their own journeymen, and were referred to as “Masterpieces”.

We use the term a bit more loosely these days, especially when coupled with the word “theatre”, but the sense remains. A Masterpiece is a piece of work that demonstrates your mastery of the craft and any sensible group of experts within your discipline would recognise it as such and declare you worthy to join them.

On reflection, after my last post on identity, I realised that I had placed the PhD into a very specific place, based on the PhD culture of my own discipline and my own experience. There are people who work their way up through a discipline for years, advancing steadily through their craft via diploma, recognition of prior learning and finally degree. Finally, having functioned as practitioner, they move into the academy in order to make their definitive contribution and it is as practitioner-academics that they create their final thesis which, in some regard, has more than a hint of the mastery of the craft about it and is far more likely to be a masterpiece than, say, my three year musing on big systems and XML. I regard myself more as an academic-practitioner as while I have previous knowledge, my research work began afresh and my PhD formed the basis of my qualification for entry into the profession of academic (journeyman) rather than the condensation of my life’s contribution as a practitioner, placed within the academic sphere to change teaching, research and policy (masterpiece).

However, this really doesn’t clear the issue up at all, all it does is emphasise that it is the recognition of the masterpiece that determines one’s mastery, which in turn requires that we have strong “guilds” or their equivalent in order to be able to clearly state when something has been produced to a level that we have met this particular skill battier.

Now, in terms of supervising other PhD students, I can do that now but, until my first student completes successfully (fingers crossed for December), I cannot be a principal supervisor. I am apprenticed, again, in effect until I have demonstrated sufficient mastery. So my PhD qualification is, again, rendered at the journeyman level. If I still had my network certifications from my previous life, I could instruct people in networking within certain corporate frameworks, but I (again) only had journeyman qualifications here. I have a friend who has achieved mastery in the networking discipline and the difference in our skill levels is amazing but, rather sadly, he has no masterpiece to show for his efforts. He worked to solve some difficult problems, and sat some very hard exams, and provided that he repeats this performance every 2 years, he will make lots of money doing interesting things involving networks. There is not, however, a single artefact of his that he can point to, which asserts that from that point on, he had mastery of a certain set of skills.

And this is very much the way of modern mastery. Why does my friend have to resit his exams? Because things are changing very quickly these days and, because of the Internet, we can propagate those changes almost immediately. A master craftsman of the 17th Century would learn new techniques, certainly, but having achieved mastery, he would enjoy maybe 20-30 more years of relatively low change until he died of some unspeakable disease or a falling giraffe. These days, while master craftsman certainly exist and are recognised as such, in many scientific disciplines, we tend to award this towards the end of someone’s life, at a time when their practical life is relatively close to over and I wonder if that is to stop the embarrassment of a recognised master who knows nothing about what has happened in the field because it has all moved on.

How do we recognise mastery in science, literature or academia? Well, there are significant Fellowships (the Royal Society springs to mind), important prizes (the Nobel, the Pulitzer) and awards (the Turing and the like). Of course, there is one award that recognises early achievement, the Fields Medal in mathematics, which may only be awarded to someone who is not yet 40, specifically to try and encourage the recipients to go further and do more. A lot of these awards and prizes, however, allow the luxury of a Masterpiece, especially those awards which are given for a specific piece of work. But which of J. M. Coetzee’s works was the definitive masterpiece that granted him the Nobel in Literature, the one that tipped the balance? Where is the specific masterpiece that I can pass to other guild members (not that I am one) and admire, wish that I had created, and learn from? Even where we have the books, we still don’t have a clear notion of what we are looking at. (I realise that Coetzee’s skills were clearly identified in the award, as well as his focus, and I am certainly not disputing the validity – but which is the book I give to someone to explain why he is a master?)

It is much harder to see where we give our students the ability to produce master works of any kind, even within our capstone courses. The works produced under capstone are more likely to be fit-for-purpose, complete but unremarkable, and therefore fit to judge for the end of apprenticeship, but no further. If they then progress to Honours, Masters or PhD, they do not so much have an opportunity to produce a masterpiece, what they are doing is conducting an apprenticeship for a new trade. (This varies by profession and intent. I can quite happily see that a PhD in Creative Writing has a masterpiece component attached to it, whereas a PhD in other disciplines may not.)

But, given that the international recognition of mastery is in a highly refined atmosphere and can, at most, accommodate a very small number of people, how do we even recognise those few masterpieces that will occur outside of the defining masterworks of a generation? For me, as a personal reflection, I am coming to terms with the fact that any masterpiece that I do produce, a work of great import or even a student (in some respects) that goes on to change the world, may have a very short shelf-life compared to other crafts. I also have to accept that the guild that accepts it as master work may never even contact me to tell me what they think – I’ll just have to watch my citation index go up and use it to get myself promoted.

I don’t have a complete answer to this, and I know that there’s a lot more thinking to do, but are we looking at the end of masterpieces or do we just have to adopt a different lens for seeing them, as well as a different group for judging them?