Let’s Transform Education! (MOOC Hijinks Hilarity! Jinkies!)

Posted: September 1, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , | 4 Comments

I had one of those discussions yesterday that every one in Higher Education educational research comes to dread: a discussion with someone who basically doesn’t believe the educational research and, within varying degrees of politeness, comes close to ignoring or denigrating everything that you’re trying to do. Yesterday’s high point was the use of the term “Mr Chips” to describe the (from the speaker’s perspective) incredibly low possibility of actually widening our entrance criteria and turning out “quality” graduates – his point was that more students would automatically mean much larger (70%) failure rates. My counter (and original point) is that since there is such a low correlation between school marks and University GPA (roughly 40-45% and it’s very noisy) that successful learning and teaching strategies could deal with an influx of supposedly ‘lower quality’ students, because the quality metric that we’re using (terminal high school grade or equivalent) is not a reliable indicator of performance. My fundamental belief is that good education is transformative. We start with the students that schools give us but good, well-constructed, education can, in the vast majority of cases, successfully educate students and transform them into functioning, self-regulating graduates. We have, as a community, carried out a lot of research that says that this works, provided that we are happy to accept that we (academics) are not by any stretch of the imagination the target demographic or majority experience in our classes, and that, please, let’s look at new teaching methods and approaches that actually work in developing the knowledge and characteristics that we’re after.

The “Mr Chips” thing is a reference to a rather sentimental account of the transformative influence of a school master, the eponymous Chips, and, by inference, using it in a discussion of the transformative power of education does cast the perception of my comments on equality of access, linked with educational design and learning systems as transformative technologies, as being seen as both naïve and (in a more personal reading) makes me uncomfortably aware that some people might think I’m talking about myself as being the key catalyst of some sort. One of the nice things about being an academic is that you can have a discussion like this and not actually come to blows over it – we think and argue for a living, after all. But I find this dismissive and rude. If we’re not trying to educate people and transform them, then what the hell are we doing? Advocating inclusion and transformation shouldn’t be seen as grandstanding – it should be seen as our job. I don’t want to be the keystone, I want systems that work and survive individuals, but that individuals can work within to improve and develop – we know this is possible and it’s happening in a lot of places. There are, however, pockets of resistance: people who are using the same old approaches out of laziness, ignorance and a refusal to update for what appear to be philosophical reasons but have no evidence to support them.

Frankly, I’m getting a little irritated by people doubting the value of the volumes of educational research. If I was dealing with people who’d read the papers, I’d be happier, but I’m often dealing with people who won’t read the papers because they just don’t believe that there’s a need to change or they refuse to accept what is in there because of a perceived difficulty in making it work. (A colleague demanded a copy of one of our papers showing the impact of our new approaches on retention – I haven’t heard from him since he got it. This probably means that he’s chosen to ignore it and is going to pretend that he never asked.) Over coffee this morning, musing on this, it occurred to me that at the same time that we’re not getting the greatest amount of respect and love in the educational research community, we’re also worried about the trend towards MOOCs. Many of our concerns about MOOCs are founded in the lack of evidence that they are educationally effective. And I saw a confluence.

All of the educational researchers who are not able to sway people inside their institutions – let’s just ignore them and surge into the MOOCs. We can still teach inside our own places, of course, and since MOOCs are free there’s no commercial conflict – but let’s take all of the research and practice and build a brave new world out in MOOC space that is the best of what we know. We can even choose to connect our in-house teaching into that system if we want. (Yes, we still have the face-to-face issue for those without a bricks-and-mortar campus, but how far could we go to make things better in terms of what MOOCs can offer?) We’re transformers, builders and creators. What could we do with the infinite canvas of the Internet and a lot of very clever people, working with a lot of very other clever people who are also driven and entrepreneurial?

The MOOC community will probably have a lot to say about this, which is why we shouldn’t see this as a hijack or a take-over, and I think it’s helpful to think of this very much as a confluence – a flowing together. I am, not for a second, saying that this will legitimise MOOCs, because this implies that they are illegitimate, but rather than keep fighting battles with colleagues and systems that can defeat 40 years of knowledge by saying “Well, I don’t think so”, let’s work with people who have already shown that they are looking to the future. Perhaps, combining people who are building giant engines of change with the people who are being frustrated in trying to bring about change might make something magical happen? I know that this is already happening in some places – but what if it was an international movement across the whole sector?

Jinkies! (Sorry, the title ran to this and I get to use a picture of a t-shirt with Velma on it!)

Relma!

The purpose of this is manifold:

  1. We get to build the systems that we want to, to deliver education to students in the best ways we know.
  2. We (potentially) help to improve MOOCs by providing strong theory to construct evidence gathering mechanisms that allow us to really get inside what MOOCs are doing.
  3. More students get educated. (Ok, maybe not in our host institutions, but what is our actual goal anyway?)
  4. We form a strong international community of educational researchers with common outputs and sharing that isn’t necessarily owned by one company (sorry, iTunesU).
  5. If we get it right, students vote with their feet and employers vote with their wallets. We make educational research important and impossible to ignore through visible success.

Now this is, of course, a pipe dream in many ways. Who will pay for it? How long will it take before even not-for-pay outside education becomes barred under new terms and conditions? Who will pay my mortgage if I get fired because I’m working on a deliberately external set of courses for students who are not paying to come to my institution?

But, the most important thing, for me, is that we should continue what has been proposed and work more and more closely with the MOOC community to develop exemplars of good practice that have strong, evidence-based outcomes that become impossible to ignore. Much as students use temporal discounting to procrastinate about their work, administrators tend to use a more traditional financial discounting when it comes to what they consider important. If it takes 12 papers and two years of study to justify spending $5,000 on a new tool or time spent on learning design – forget about it. If, however, MOOCs show strong evidence of improving student retention (*BING*), student attraction (*BING*), student engagement (*BING*) and employability – well, BINGO. People will pay money for that.

I’ve spoken before about how successful I had to be before I was tolerated in my pursuit of educational research and, while I don’t normally talk about it in detail because it smacks of hubris and I sincerely believe that I am not a role model of any kind, I  hope that you will excuse me so that I can explain why I think it’s crazy as to how successful I had to be in order to become tolerated – and not yet really believed. To summarise, I’m in three research groups, I’ve brought in (as part of a group and individually) somewhere in the order of $0.5M in one non-ed research area, I’ve brought in something like $30-50K in educational research money, I’ve published two A journals (one CS research, one CS ed), two A conferences (both ed) and one B conference (ed/CS) and I have a faculty level position as an Associate Dean and I have a national learning and teaching presence. All of the things on that line – that’s 2012. 2011 wasn’t quite as successful but it wasn’t bad by any stretch of the imagination. I think that’s an unreasonably high bar to pass in order to be allowed the luxury of asking questions about what it is that we’re doing with learning and teaching. But if I can leverage that to work with other colleagues who can then refer to what we’ve done in a way that makes administrators and managers accept the real value of an educational revolution – then my effort is shared over many more people and it suddenly looks like a much better investment of my time.

This is more musing that mission, I’m afraid, and I realise that any amount of this could be shot down but I look forward to some discussion!

 

 

More Thoughts on Partnership: Teacher/Student

Posted: August 23, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , , , | Leave a comment

I’ve just received some feedback on an abstract piece that is going into a local educational research conference. I talked about the issues with arbitrary allocation of deadlines outside of the framing of sound educational design and about how it fundamentally undermines any notion of partnership between teacher and student. The responses were very positive although I’m always wary when people staring using phrases like “should generate vigorous debate around expectations of academics” and “It may be controversial, but [probably] in a good way”. What interests me is how I got to the point of presenting something that might be considered heretical – I started by just looking at the data and, as I uncovered unexpected features, I started to ask ‘why’ and that’s how I got here.

When the data doesn’t fit your hypothesis, it’s time to look at your data collection, your analysis, your hypothesis and the body of evidence supporting your hypothesis. Fortunately, Bayes’ Theorem nicely sums it up for us: your belief in your hypothesis after you collect your evidence is proportional to how strongly your hypothesis was originally supported, modified by the chances of seeing what you did given the existing hypothesis. If your data cannot be supported under your hypothesis – something is wrong. We, of course, should never just ignore the evidence as it is in the exploration that we are truly scientists. Similarly, it is in the exploration of our learning and teaching, and thinking about and working on our relationship with our students, that I feel that we are truly teachers.

All your Bayes are belong to us. (Sorry.)

Once I accepted that I wasn’t in competition with my students and that my role was not to guard the world from them, but to prepare them for the world, my job got easier in many ways and infinitely more enjoyable. However, I am well aware that any decisions I make in terms of changing how I teach, what I teach or why I teach have to be based in sound evidence and not just any warm and fuzzy feelings about partnership. Partnership, of course, implies negotiation from both sides – if I want to turn out students who will be able to work without me, I have to teach them how and when to negotiate. When can we discuss terms and when do we just have to do things?

My concern with the phrase “everything is negotiable” is that it, to me, subsumes the notions that “everything is equivalent” and “every notion is of equal worth”, neither of which I hold to be true from a scientific or educational perspective. I believe that many things that we hold to be non-negotiable, for reasons of convenience, are actually negotiable but it’s an inaccurate slippery slope argument to assume that this means that we  must immediately then devolve to an “everything is acceptable” mode.

Once again we return to authenticity. There’s no point in someone saying “we value your feedback” if it never shows up in final documents or isn’t recorded. There’s no point in me talking about partnership if what I mean is that you are a partner to me but I am a boss to you – this asymmetry immediately reveals the lack of depth in my commitment. And, be in no doubt, a partnership is a commitment, whether it’s 1:1 or 1:360. It requires effort, maintenance, mutual respect, understanding and a commitment from both sides. For me, it makes my life easier because my students are less likely to frame me in a way that gets in the way of the teaching process and, more importantly, allows them to believe that their role is not just as passive receivers of what I deign to transmit. This, I hope, will allow them to continue their transition to self-regulation more easily and will make them less dependent on just trying to make me happy – because I want them to focus on their own learning and development, not what pleases me!

One of the best definitions of science for me is that it doesn’t just explain, it predicts. Post-hoc explanation, with no predictive power, has questionable value as there is no requirement for an evidentiary standard or framing ontology to give us logical consistency. Seeing the data that set me on this course made me realise that I could come up with many explanations but I needed a solid framework for the discussion, one that would give me enough to be able to construct the next set of analyses or experiments that would start to give me a ‘why’ and, therefore, a ‘what will happen next’ aspect.

 

Wading In: No Time For Paddling

Posted: July 31, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , , | 1 Comment

I’m up to my neck in books on visualisation and data analysis at the moment. So up to my neck that this post is going to be pretty short – and you know how much I love to talk! I’ve spent most of the evening preparing for tomorrow’s visualising data tutorial for Grand Challenges and one of the things I was looking for was bad visualisations. I took a lot away from Mark’s worked examples posts, and I look forward to seeing the presentation, but visualisation is a particularly rich area for worked ‘bad’ examples. With code, it has to work to a degree or manifest its failure in interesting ways. A graphic can be completely finished and still fail to convey the right information.

(I’ve even thrown in some graphics that I did myself and wasn’t happy with – I’m looking forward to the feedback on those!) (Ssh, don’t tell the students.)

I had the good fortune to be given a copy of Visual Strategies (Frankel and DePace) which was designed by one of the modern heroes of design – the amazing Stefan Sagmeister. This is, without too much hyperbole, pretty much the same as being given a book on painting where Schiele had provided the layout and examples. (I’m a very big fan of Egon Schiele and Hundertwasser for that matter. I may have spent a little too much time in Austria.) The thing I like about this book is that it brings a lot of important talking and thinking points together: which questions should you ask when thinking about your graphic, how do you start, what do you do next, when do you refine, when do you stop?

Thank you, again, Metropolis Bookstore on Swanston Street in Melbourne! You had no real reason to give a stranger a book for free, except that you thought it would be useful for my students. It was, it is, and I thank you again for your generosity.

I really enjoy getting into a new area and I think that the students are enjoying it too, as the entire course is a new area for them. We had an excellent discussion of the four chapters of reading (the NSF CyberInfrastructure report on Grand Challenges), where some of it was a critique of the report itself – don’t write a report saying “community engagement and visualisation are crucial” and (a) make it hard to read, even for people inside the community or (b) make it visually difficult to read.

On the slightly less enthusiastic front, we get to the crux of the course this week – the project selection – and I’m already seeing some hesitancy. Remember that these are all very good students but some of them are not comfortable picking an area to do their analysis in. There could be any number of reasons so, one on one, I’m going to ask them why. If any of them say “Well, I could if I wanted to but…” then I will expect them to go and do it. There’s a lot of scope for feedback in the course so an early decision that doesn’t quite work out is not a death sentence, although I think that waiting for permission to leap is going to reduce the amount of ownership and enjoyment that the student feels when the work is done.

I have no time for paddling in the shallows, personally, and I wade on in. I realise, however, that this is a very challenging stance for many people, especially students, so while I would prefer people to jump in, I recognise my job as life guard in this area and I am happy to help people out.

However, these students are the Distinction/High Distinction crowd, the ones who got 95-100 on leaving secondary school and, as we thought might occur, some of them are at least slightly conditioned to seek my approval, a blessing for their project choice before they have expended any effort. Time to talk to people and work with them to help them move on to a more confident and committed stance – where that confidence is well-placed and the commitment is based on solid fact and thoughtful reasoning!

 

Reflecting on rewards – is Time Banking a reward or a technique?

Posted: July 29, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , | Leave a comment

The Reward As It Is Often Implemented
(In Advance and Starting With a B)

Enough advocacy for a while, time to think about research again! Given that I’ve just finished Alfie Kohn’s Punished by Rewards, and more on that later, I’ve been looking very carefully at everything I do with students to work out exactly what I am trying to do. One of Kohn’s theses is that we tend to manipulate people towards compliance through extrinsic tools such as incentives and rewards, rather than provide an environment in which their intrinsic motivational aspects dominate and they are driven to work through their own interest and requirements. Under Kohn’s approach, a gold star for sitting quietly achieve little except to say that sitting quietly must be so bad that you need to be bribed, and developing a taste for gold stars in the student. If someone isn’t sitting quietly, is it because they haven’t managed sitting quietly (the least rewarding unlockable achievement in any game) or that they are disengaged, bored or failing to understand why they are there? Is it, worse, because they are trying to ask questions about work that they don’t understand or because they are so keen to discuss it that they want to talk? Kohn wants to know WHY people are or aren’t doing things rather than just to stop or start people doing things through threats and bribery.

Where, in this context, does time banking fit? For those who haven’t read me on this before, time banking is described in a few posts I’ve made, with this as one of the better ones to read. In summary, students who hand up work early (and meet a defined standard) get hours in the bank that they can spend at a later date to give themselves a deadline extension – and there are a lot of tuneable parameters around this, but that’s the core. I already have a lot of data that verifies that roughly a third of students hand in on the last day and 15-18% hand up late. However, the 14th highest hand-in hour is the one immediately after the deadline. There’s an obvious problem where people aren’t giving themselves enough time to do the work but “near-missing” by one hour is a really silly way to lose marks. (We won’t talk about the pedagogical legitimacy of reducing marks for late work at the moment, that’s a related post I hope to write soon. Let’s assume that our learning design requires that work be submitted at a certain time to reinforce knowledge and call that the deadline – the loss, as either marks or knowledge reinforcement, is something that we want to avoid.)

But, by providing a “reward” for handing up early, am I trying to bribe my students into behaviour that I want to see? I think that the answer is “no”, for reasons that I’ll go into.

Firstly, the fundamental concept of time banking is that students have a reason to look at their assignment submission timetable as a whole and hand something up early because they can then gain more flexibility later on. Under current schemes, unless you provide bonus points, there is no reason for anyone to hand up more than one second early – assuming synchronised clocks. (I object to bonus points for early hand-in for two reasons: it is effectively a means to reward the able or those with more spare time, and because it starts to focus people on handing up early rather than the assignment itself.) This, in turn, appears to lead to a passive, last minute thinking pattern and we can see the results of that in our collected assignment data – lots and lots of near-miss late hand-ins. Our motivation is to focus the students on the knowledge in the course by making them engage with the course as a whole and empowering themselves into managing their time rather than adhering to our deadlines. We’re not trying to control the students, we’re trying to move them towards self-regulation where they control themselves.

Secondly, the same amount of work has to be done. There is no ‘reduced workload’ for handing in early, there is only ‘increased flexibility’. Nobody gets anything extra under this scheme that will reinforce any messages of work as something to be avoided. The only way to get time in the bank is to do the assignments – it is completely linked to the achievement that is the core of the course, rather than taking focus elsewhere.

Thirdly, a student can choose not to use it. Under almost every version of the scheme I’ve sketched out, every student gets 6 hours up their sleeve at the start of semester. If they want to just burn that for six late hand-ins that are under an hour late, I can live with that. It will also be very telling if they then turn out to be two hours late because, thinking about it, that’s a very interesting mental model that they’ve created.

But how is it going to affect the student? That’s a really good question. I think that, the way that it’s constructed, it provides a framework for students to work with, one that ties in with intrinsic motivation, rather than a framework that is imposed on students – in fact, moving away from the rigidly fixed deadlines that (from our data) don’t even to be training people that well anyway is a reduction in manipulate external control.

Will it work? Oh, now, there’s a question. After about a year of thought and discussion, we’re writing it all up at the moment for peer review on the foundations, the literature comparison, the existing evidence and the future plans. I’ll be very interested to see both the final paper and the first responses from our colleagues!

The Early-Career Teacher

Posted: July 24, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Recently, I mentioned the Australian Research Council (ARC) grant scheme, which recognises that people who have had their PhDs for less than five years are regarded as early-career researchers (ECRs). ECRs have a separate grant scheme (now, they used to have a different way of being dealt with in the grant application scheme) that recognises the fact that their track records, the number of publications and activity relative to opportunity, is going to be less than that of more seasoned individuals.

What is interesting about this is that someone who has just finished their PhD will have spent (at least) three years, more like four, doing research and, we hope, competent research under guidance for the last two of those years. So, having spent a couple of years doing research, we then accept that it can take up to five years for people to be recognised as being at the same level.

But, for the most part, there is no corresponding recognition of the early-career teacher, which is puzzling given that there is no requirement to meet any teaching standards or take part in any teaching activities at all before you are put out in front of a class. You do no (or are not required to do any) teaching during your PhD in Australia, yet we offer support and recognition of early status for the task that you HAVE been doing – and don’t have a way to recognise the need to build up your teaching.

We discussed ideas along these lines at a high-level meeting that I attended this morning and I brought up the early-career teacher (and mentoring program to support it) because someone had brought up a similar idea for researchers. Mentoring is very important, it was one of the big HERDSA messages and almost everywhere I go stresses this, and it’s no surprise that it’s proposed as a means to improve research but, given the realities of the modern Australian University where more of our budget comes from teaching than research, it is indicative of the inherent focus on research that I need to propose teaching-specific mentoring in reaction to research-specific mentoring, rather than vice versa.

However, there are successful general mentoring schemes where senior staff are paired with more junior staff to give them help with everything that they need and I quite like this because it stresses the nexus of teaching and research, which is supposed to be one of our focuses, and it also reduces the possibility of confusion and contradiction. But let’s return to the teaching focus.

The impact of an early-career teacher program would be quite interesting because, much as you might not encourage a very raw PhD to leap in with a grant application before there was enough supporting track record, you might have to restrict the teaching activities of ECTs until they had demonstrated their ability, taken certain courses or passed some form of peer assessment. That, in any form, is quite confronting and not what most people expect when they take up a junior lectureship. It is, however, a practical way to ensure that we stress the value of teaching by placing basic requirements on the ability to demonstrate skill within that area! In some areas, as well as practical skill, we need to develop scholarship in learning and teaching as well – can we do this in the first years of the ECT with a course of educational psychology, discipline educational techniques and practica to ensure that our lecturers have the fundamental theoretical basis that we would expect from a school teacher?

Are we dancing around the point and, extending the heresy, require something much closer to the Diploma of Education to certify academics as teachers, moving the ECR and the ECT together to give us an Early Career Academic (ECA), someone who spends their first three years being mentored in research and teaching? Even ending up with (some sort of) teaching qualification at the end? (With the increasing focus on quality frameworks and external assessment, I keep waiting for one of our regulatory bodies to slip in a ‘must have a Dip Ed/Cert Ed or equivalent’ clause sometime in the next decade.)

To say that this would require a major restructure in our expectations would be a major understatement, so I suspect that this is a move too far. But I don’t think it’s too much to put limits on the ways that we expose our new staff to difficult or challenging teaching situations, when they have little training and less experience. This would have an impact on a lot of teaching techniques and accepted practices across the world. We don’t make heavy use of Teaching Assistants (TAs) at my Uni but, if we did, a requirement to reduce their load and exposure would immediately push more load back onto someone else. At a time when salary budgets are tight and people are already heavily loaded, this is just not an acceptable solution – so let’s look at this another way.

The way that we can at least start this, without breaking the bank, is to emphasise the importance of teaching and take it as seriously as we take our research: supporting and developing scholarship, providing mentoring and extending that mentoring until we’re sure that the new educators are adapting to their role. These mentors can then give feedback, in conjunction with the staff members, as to what the new staff are ready to take on. Of course, this requires us to carefully determine who should be mentored, and who should be the mentor, and that is a political minefield as it may not be your most senior staff that you want training your teachers.

I am a fairly simple man in many ways. I have a belief that the educational role that we play is not just staff-to-student, but staff-to-staff and student-to-student. Educating our new staff in the ways of education is something that we have to do, as part of our job. There is also a requirement for equal recognition and support across our two core roles: learning and teaching, and research. I’m seeing a lot of positive signs in this direction so I’m taking some heart that there are good things on the nearish horizon. Certainly, today’s meeting met my suggestions, which I don’t think were as novel as I had hoped they would be, with nobody’s skull popping out of their mouth. I take that as a positive sign.

 

A Design Challenge, a Grand Design Challenge, if you will.

Posted: July 18, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , | 1 Comment

Question: What is one semester long, designed as a course for students who perform very well academically, has no prerequisites and can be taken by students with no programming exposure and by students with a great deal of programming experience?

Answer: I don’t know but I’m teaching it on Monday.

While I talk about students who perform well academically, this is for the first instance of this course. My goal is that any student can take this course, in some form, in the future.

The new course in our School, Grand Challenges in Computer Science, is part of our new degree structure, the Bachelor of Computer Science (Advanced). This adds  lot more project work and advanced concepts, without disrupting the usual (and already excellent) development structure of the degree. One of the challenges of dealing with higher-performing students is keeping them in a sufficiently large and vibrant peer group while also addressing the minor problem that they’re moving at a different pace to many people that they are friends with. Our solution has been to add additional courses that sit outside of the main progression but still provide interesting material for these students, as well as encouraging them to take a more active role in the student and general community. They can spend time with their friends, carry on with their degrees and graduate at the same time, but also exercise themselves to greater depth and into areas that we often don’t have time to deal with.

In case you’re wondering, I know that some of my students read this blog and I’m completely comfortable talking about the new course in this manner because (a) they know that I’m joking about the “I don’t know” from the Answer above and (b) I have no secrets regarding this course. There are some serious challenges facing us as a species. We are now in a position where certain technologies and approaches may be able to help us with this. One of these is the notion of producing an educational community that can work together to solve grand challenges and these students are very much a potential part of this new community.

The biggest challenge for me is that I have such a wide range of students. I have students who potentially have no programming background and students who have been coding for four years. I have students who are very familiar with the School’s practices and University, and people whose first day is Monday. Of course, my solution to this is to attack it with a good design. But, of course, before a design, we have to know the problem that we’re trying to solve.

The core elements of this course are the six grand challenges as outlined but he NSF, research methods that will support data analysis, the visualisation of large data sources as a grand challenge and community participation to foster grand challenge communities. I don’t believe that a traditional design of lecturing is going to support this very well, especially as the two characteristics that I most want to develop in the students are creativity and critical thinking. I really want all of my students to be able to think their way around, over or through an obstacle and I think that this course is going to be an excellent place to be able to concentrate on this.

I’ve started by looking at my learning outcomes for this course – what do I expect my students to know by the end of this course? Well, I expect them to be able to tell me what the grand challenges are, describe them, and then provide examples of each one. I expect them to be able to answer questions about key areas and, in the areas that we explore in depth, demonstrate this knowledge through the application of relevant skills, including the production of assignment materials to the best of their ability, given their previous experience. Of course, this means that every student may end up performing slightly differently, which immediately means that personalised assessment work (or banded assessment work) is going to be required but it also means that the materials I use will need to be able to support a surface reading, a more detailed reading and a deep reading, where students can work through the material at their own pace.

I don’t want the ‘senior’ students to dominate, so there’s going to have be some very serious scaffolding, and work from me, to support role fluidity and mutual respect, where the people leading discussion rotate to people supporting a point, or critiquing a point, or taking notes on the point, to make sure that everyone gets a say and that we don’t inhibit the creativity that I’m expecting to see in this course. I will be setting standards for projects that take into account the level of experience of each person, discussed and agreed with the student in advance, based on their prior performance and previous knowledge.

What delights me most about this course is that I will be able to encourage people to learn from each other. Because the major assessment items are all unique to a student, then sharing knowledge will not actually lead to plagiarism or copying. Students will be actively discouraged from doing work for each other but, in this case, I have no problem in students helping each other out – as long as the lion’s share of the work is done by the main student. (The wording of this is going to look a lot more formal but that’s a Uni requirement. To quote “The Castle”, “It’s about the vibe.”) Students will regularly present their work for critique and public discussion, with their response to that critique forming a part of their assessment.

I’m trying to start these students thinking about the problems that are out there, while at the same time giving them a set of bootstrapping tools that can set them on the path to investigation and (maybe) solution well ahead of the end of their degrees. This then feeds into their project work in second and third year. (And, I hope, for at least some of them, Honours and maybe PhD beyond.)

Writing this course has been a delight. I have never had so much excuse to buy books and read fascinating things about challenging issues and data visualisation. However, I think that it will be the student’s response to this that will give me something that I can then share with other people – their reactions and suggestions for improvement will put a seal of authenticity on this that I can then pack up, reorganise, and put out into the world as modules for general first year and high school outreach.

I’m very much looking forward to Monday!

Good Design: Building In Important Features From the Start

Posted: July 17, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , | Leave a comment

The game “Deus Ex” is widely regarded as one of the best computer games that has been made so far. It has won a very large number of “best game” awards and regularly shows up in the top 5 of lists of “amazing games”. Deus Ex was released in 2000, designed and developed by Ion Storm under Warren Spector and Harvey Smith and distributed by Eidos. (I mentioned it before in this post, briefly.) Here is the description of this game from Wikipedia:

Set in a dystopian world during the year 2052, the central plot follows rookie United Nations Anti-Terrorist Coalition agent JC Denton, as he sets out to combat terrorist forces, which have become increasingly prevalent in a world slipping ever further into chaos. As the plot unfolds, Denton becomes entangled in a deep and ancient conspiracy, encountering organizations such as Majestic 12, the Illuminati, and the Hong Kong Triads throughout his journey.

Deus Ex had a cyberpunk theme, a world of shadowy corporations and many corruptions of the human soul, ranging from a generally materialistic culture to body implants producing cyborg entities that no longer had much humanity. While looking a lot like a First-Person Shooter (you see through the character’s eyes and kill things), the game also had a great deal of stealth play (sneaking around trying very hard not to get noticed, shot or both). However, what sets DE apart from most other games it that the choice of how you solved most of the problems was pretty much left up to you. This was no accident. The fact that you could solve 99% of the problems in the game by using different forms of violence, many forms of stealth or a combination of these was down to the way that the game was designed.

When I was at Game Masters at ACMI, Melbourne, over the weekend, I was able to read the front page of a document entitled “Just What IS Deus Ex” by Warren Spector. Now, unfortunately, they had a “no photographs” rule so I don’t have a copy of it (and, for what it’s worth, I also interpreted that to mean “no tiresome hand transcription onto the iPhone in order to make a replica” ) but one of the most obvious and important design features was that they wanted to be able to support player exploration: players’ actions had to have consequences and players needed to be able to make their plans, without feeling constrained by the world. (Fortunately, while not being the actual document, there is an article here where Warren talks about most of the important things. If you’re interested in design, have a look at it after you’ve finished this.) Because of this, a number of the items in the game can be used in a number of quite strange ways and, while it appears that this is a bug, suddenly you’ll run across an element of the game that makes you realise that the game designers knew that this was possible.

Do not climb if red lights active!

For example, in the Triad-run Hong Kong of 2052, there is a very tall tower on one edge of the explorable area. There are grenades (LAMs)in the game that adhere ‘magnetically’ to walls and then explode if armed and someone enters their proximity. However, it is possible to use these grenades to climb up walls, assuming you don’t arm them of course, by sticking them to walls, getting close enough to hop up, placing another grenade above you and then doing the same thing. With patience, you can climb quite high. Sounds like a bug, right? Yeah, well, that’s what I thought until I climbed to the top of the tower in Hong Kong and found a guy, one of the Non-Player Characters, standing on top.

This was a surprise but it shouldn’t have been. I’d already realised that there was always more than one way to do things and, because the game was designed to make it is as easy as possible for me to try many paths to achieve success, the writer had put in early hints designed to discourage a ‘blow everything up’ approach. The skill system makes it relatively easy for you to make your life a lot easier by working with what is already in the environment rather than trying to do it all yourself.

In terms of the grenades, rather than just being pictures on a wall, they became real world objects when placed and were as solid as any other element. This allowed them to be climbed and the designers/programmers recognised this by putting a guy on top of a tower that you had no other way to get to (without invoking cheats). The objects in Deus Ex were designed to be as generally usable as possible. The sword could open crates as well (Ok, well much better) than a crowbar could and reduced the need to carry two things. Many weapons came with multiple ammunition types, allowing you to customise your load out to the kind of game you wanted to play. Other nice features included the fact that there very few situations of ‘spontaneous creation’, where monsters appeared at some point in a scripted scene, which would have enforced a certain approach. If you were crawling in somewhere from completely the wrong side, everything would be there and ready, rather than all spontaneously reappearing when you happened to approach from the ‘triggering’ side.

In short, it felt like a real world. (With the usual caveat regarding it being a real world where you are a killer cyborg in 2052.)

The big advantage of this is that you feel a great deal of freedom in your planning and implementation and, combined with the fact that the game reacts and changes to the decisions that you make, this makes the endings of the game feel very personal – when you finally choose between the three possible endings, you do so feeling like the game is actually going along with the persona that you have set up. This increases the level of engagement, achievement and enjoyment.

One of Mark Guzdial’s recent posts talked about the importance of good design when it comes to constructing instructional materials and I couldn’t agree more. Good design at the start, with a clear idea of what you’re trying to achieve, allows you to build a consistent experience that will allow you and your students to achieve your objectives. Deus Ex is, in my opinion, considered one of the best games of the 21st century because it started from a simple and clear design document that was set out to maximise the degree of influence that the player could feel in the game – everyone who plays Deus Ex takes their own path through it, has their own experience and gets something slightly different out of it.

I’m not saying it’s that easy for educational design as a global issue, but it is a very good reminder of why we should be doing good design at the very beginning of our courses!

Wrath of Kohn: Well, More Thoughts on “Punished by Rewards”

Posted: July 16, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , | 1 Comment

Yesterday, I was discussing my reading of Alfie Kohn’s “Punished by Rewards” and I was talking about a student focus to this but today I want to talk about the impact on staff. Let me start by asking you to undertake a quick task. Let’s say you are looking for a new job, what are the top ten things that you want to get from it? Write them down – don’t just think about them, please – and have them with you. I’ll put a picture of Kohn’s book here to stop you looking ahead. 🙂

It’s ok, I’ll wait. Written your list?

How far up the list was “Money”? Now, if you wrote money in the top three, I want you to imagine that this new job will pay you a fair wage for what you’re going to do and you won’t have any money troubles. (Bit of a reach, sometimes, I know but please give it a try.) With that in mind, look at your list again.

Does the word “excellent incentive scheme” or “great bonus package” figure anywhere on that list? If it does, is it in the top half or the bottom half? If Money wasn’t top three, where was it for you?

According to Kohn, very few people actually to make money the top of their list – it tends to be things like ‘type of work’, ‘interesting job’, ‘variety’, ‘challenge’ and stuff like that. So, if that’s the case, why do so many work incentive schemes revolve around giving us money or bonuses as a reward if, for the majority of the population, it’s not the thing that we want? Well, of course, it’s easy. Giving feedback or mentoring is much harder than a $50 gift card, a $2,000 bonus or 500 shares. What’s worse is, because it’s money, it has to be allocated in an artificial scarcity environment or it’s no longer a bonus, it’s an expectation. If you didn’t do this, then the company might go bankrupt.

What if, instead, when you did something really good, you received something that made it easier for you to do all of your work as a recognition of the fact that you’re working a lot? Of course, this would require your manager to have a really good idea of what you were doing and how to improve it, as well as your company being willing to buy you that backlit keyboard with the faux-mink armrest that will let you write reports without even a twinge of arm strain. Some of this, obviously, is part of minimum workplace standards but the idea is that you get something that reflects that your manager understands what you’ve done and is trying to help you to develop further. Carefully selected books, paid trips to useful training, opportunities to further display your skill – all of these are probably going to achieve more of the items on your 10-point list than money will. To quote Kohn, quoting Gruenberg (1980), “The Happy Worker: An Analysis of Educational and Occupational Differences in Determinants of Job Satisfaction”, American Journal of Sociology, 86, pp 267-8:

“Extrinsic rewards become an important determinant of overall job satisfaction only among workers for whom intrinsic rewards are relatively unavailable.”

There are, Kohn argues, many issues with incentive schemes as reward and one of these is the competitive environment that it fosters. I discussed this yesterday so I’ll move to one of the other, which is focusing on meeting the requirements for reward at the expense of quality and in a way that is as safe as possible. Let me give you an example that I recently encountered outside of work: Playing RockBand or SingStar (music games that score your performance). Watch me and my friends who actually sing playing a singing game: yes, we notice the score, but we don’t care about the score. We interpret, we mess around, we occasionally affect the voices of the Victorian-era female impersonator characters from Little Britain. Then watch other groups of people who are playing the game to make the highest score. They don’t interpret. They don’t harmonise spontaneously. In many cases, they barely even sing and focus on making the minimum tunefully accurate noise possible at exactly the right time, having learned the sequence, to achieve the highest score. The quality of the actual singing is non-existent, because this isn’t singing, it’s score-maximisation. Similarly, risk taking has been completely removed. (As an aside, I have excellent pitch and, to my ears, most people who try to maximise their score sound out of tune because they are within the tolerances that the game accepts, but by choosing not to actually sing, there is no fundamental thread of musicality that runs through their performance. I once saw a professional singer deliver a fantastic version of a song, only to be rated as mediocre by the system,)

On Saturday, my wife and I went to the Melbourne-based Australian Centre for the Moving Image (ACMI) to attend the Game Masters gaming exhibition. It was fantastic, big arcade section and tons of great stuff dedicated to gaming. (Including the design document for Deus Ex!) There were lots of games to play, including SingStar (Scored karaoke), RockBand (multi-instrument band playing with feedback and score) and some dancing games. Going past RockBand, Katrina pointed out how little fun the participants appeared to be having and, on looking at it, it was completely true. The three boys in there were messing around with pseudo-musical instruments but, rather than making a loud and joyful noise, they were furrowed of brow and focused on doing precisely the right things at the right times to get positive feedback and a higher score. Now, there are many innovations emerging in this space and it is now possible to explore more and actually take some risks for innovation, but from industry and from life experience, it’s pretty obvious that your perception of what you should be doing and where the reward is going to come from make a huge difference.

If your reward is coming from someone/something else, and they set a bar of some sort, you’re going to focus on reaching that bar. You’re going to minimise the threats to not reaching that bar by playing it safe, colouring inside the lines, trying to please the judge and then, if you don’t get that reward, you’re far more likely to stop carrying out that activity, even if you loved it before. And, boy, if you don’t get that reward, will you feel punished.

I’m not saying Kohn is 100% correct, because frankly I don’t know and I’m not a behaviourist, but a lot of this rings true from my own experience and his use of the studies included in his book, as well as the studies themselves, are very persuasive. I look forward to some discussion on these points!

Thomas Pynchon on Education, as Quoted in Playboy Magazine

Posted: July 12, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , , | Leave a comment

I greatly enjoy reading and I read fairly widely. There are books that I enjoy more than others, certainly, but it’s rare that I find a book that doesn’t have something to teach me: in terms of conveyed knowledge, shared experience or the importance of editing. (Sorry, couldn’t resist.) I was thinking about the works of Thomas Pynchon recently and some of the quotes that we have from the author himself, as well as those contained within the books, and one that sticks out for me was:

Why should things be easy to understand?

(Pynchon to Jules Siegel regarding the complexity of “V”, from a Playboy interview, 1977)

“Run away with me.” said Roseman when the coffee came.
“Where?” she asked. That shut him up.
(The Crying of Lot 49, 1966)

It’s a very good question and one I think about a lot. “Too clever by half” and “too difficult to understand” are very easy ways to dismiss people or things that you don’t like. From an educator’s viewpoint, this is a constant hurdle that we have to leap across, especially if the students in our care have only been exposed to the easily digestible up until now. I’ve talked before about dependence on a single point of authority, which will give you all the answers in due time, and it’s not a great way to train critical thinkers – of course, it’s the antithesis.

Most things are not easy to understand, which is why teachers exist. If we could have solved the transfer of knowledge problem the moment we set stylus to clay on the banks of the Tigris, then we would have done so. Our history is full of easy discoveries of things that just happen, as well as the more complex that required diligence and sacrifice of effort. Grapes turn into wine in the presence of oxygen – the wine industry was going to happen the moment the first time a goatherd noticed her goats falling over. Knowledge does not flow from person to person as easily. There are internal barriers to deal with in transmission from you to the world, then the mutagenic ether of knowledge transmission from person to person, and finally the barriers inside the head of the receiver.

If it was going to happen, then perhaps it would have happened when we developed libraries. You could go in and browse the collected thoughts of generations. Yet, we still needed teachers and educational institutions and, while it is easy to say that this is a requirement for certification and the associated authorities, the successful person has not needed an armful of qualifications and parchments until relatively recently, so while we are suffering from a deluge of over-dependence on certification, I don’t see this as the leading justification for the role of the teacher in knowledge transfer.

One of the critical roles of the educator is to take things that are complicated and hard to understand and, with a knowledge of what the students need and their developmental stage, present the information in a way that it is comprehensible. Now, I realise that this puts me at odds, again, with people who believe that students have to struggle to attain knowledge, to demonstrate their effort and to maintain the worth of the discipline. I don’t believe that’s the role of an educator – I think that life will throw up quite enough barriers to achieving success without me force-failing someone because he or she slightly under-performs relative to other students. It’s not as if you send a child to primary school expecting that 25% of them will validly fail. (If you do, I’m both stunned and I’d love to see your reasoning!) I’ve talked with educators who are required to fail students, as part of curve grading, because their entry requirements allow anyone to come in, with any level of preparation. Those same educators are, for the most part, demoralised and unconvinced of the path that they are required to take – but they have families, and mortgages, and would like to eat tomorrow. When we walk about making things hard to understand, or not making them easier to understand, we are not only robbing the students of the joy of discovery and the thrill of legitimate achievement, we are robbing ourselves of the joy of the student who actually gets it. Yes, education is more than a linked set of “A-ha” moments but they are the sweetest fruit in a vast, ancient and ever-growing orchard.

No, I’m not (and am never) saying that anyone should automatically pass. But if someone has done what we’ve asked them to do and we have not demanded some supplication to a towering monument of obscurity in order to make them fight their way through to the facts, then I would expect a reasonably prepared student to ‘pass’ – and by that I mean take in the knowledge, incorporate it and be able to make use of it, building upon it in the future.

I, like most academics in Australia, get a lot of sample textbooks to assess for use in courses and my assessment criteria are very simple. How far can I read through the book before I get confused? How useful is the index in dealing with that confusion? Can I find the answer to a straightforward and relevant question within a couple of minutes? Can I find my way through the book?

I have four degrees, including a PhD. I am well-read and pretty literate. I have knowledge built around industry, Army, manual labour and academia. If a textbook is confusing me then it is utterly useless for my students. There is no need for things to be easy to understand but, if I am going to educate people, then I have to make sure that I put things together in a way that a reasonably prepared student can learn the knowledge that I need her or him to learn. We don’t need to require that everything be simple, we just have to remember that there is already an army of knowledge transformers, teachers, who are there in order to turn the complex into the simple for the purposes of learning.

We can retain our complexity, as long as we retain sight of our requirement to educate fairly and honestly. Pynchon’s question is ageless and still, very, valid.

The Big Picture and the Drug of Easy Understanding: Part II (Eclectic Boogaloo)

Posted: July 10, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , | Leave a comment

In yesterday’s post, I talked about the desire to place work into some sort of grand scheme, referring to movies and films, and illustrating why it’s hard to guarantee consistency from a sketch of your strategy unless you implement everything before you make it available to people. While building upon previous work is very useful, as I’m doing now, if you want to keep later works short by referring back to a shared context established in a previous work, it does make you susceptible to inconsistency if a later work makes you realise that assumptions in a previous work were actually wrong. As I noted in yesterday’s post, I’m actually writing these posts side by side and scheduling them for later, to ensure that I don’t make any more mistakes than I have to, which I can’t easily correct because the work is already displayed.

Some strategies are easier to imagine than others.

Strategic approaches to the construction of long term and complex works are essential, but a strategic plan needs to be sufficiently detailed in order to guide the works produced from it. You might get away with an abstract strategy if you produce all of the related works at one time and view them together. But, assuming that works are so long term that they can’t be produced in one sitting, you don’t want to have to seriously revise previous productions or, worse, change the strategy. This is particularly damaging when you are working with students because any significant change to the knowledge construction that you’ve been working with is going to cost you a lot of credibility and risk a high level of disengagement. Students will tolerate an amount of honest mistake, assuming that you are honest and that it is a mistake, but they tend to be very judgmental regarding poor time planning and what they perceive as laziness.

And that, in my opinion, is completely fair because we tend not to allow them poor time planning either. Going into an examination with a misunderstanding of the details of the overlying strategy will result in a non-negotiable fail, not extended understanding from the marking groups who are looking at examination performance. For me, this is an issue of professional ethics in that a consistent and fair delivery of teaching materials will facilitate learning, firstly by keeping the knowledge pathways ‘clean’ but also by establishing a relationship that you are working as hard to be fair to the student as you can, hence their effort is not wasted and you establish a bond of trust.

Now while I would love to say that this means that I have written every lecture completely before starting a new course, this would not be the truth. But this does mean that my strategic planning for new works and knowledge is broken down to a fairly fine grain plan before I start the course running. I wrote a new course last semester and the overall course had been broken up by area, sub-area, learning outcome and was built with all practicals, tutorials and activities clearly indicated. I had also spent a long time identifying the design of the overall course and the focus that we would be taking throughout, down to the structure of every lecture. When it came to writing the lectures themselves, I knew which lectures would contain ‘achievement’ items (the drug aspect where students get a buzz from the “A-ha!” moment), I knew where the pivotal points were and I’d also spent some time working out which skills I could expect in this group, and which skills later courses would expect from them.

We do have a big picture for teaching our students, in that they are part of a particular implementation of a degree that will qualify them in such-and-such a discipline. We can see the discipline syllabi, current learning and teaching practices, our local requirements and the resources that we have to carry all of this out. But this is no longer a strategy and, the more I worked with things, the more I realised that I had produced a tactical (or operational) plan for each week of the lectures – and I had to be diligent about this because one third of my lectures were being given by someone who was a new lecturer. So, on top of all the planning, every lecture had to be self-contained and instructionally annotated so that a new lecturer, with some briefing from me, could carry it out. And it all had to fit together so that structurally, semantically and stylistically, it all looked like one smooth flow.

Had I left the strategic planning to one side, in either not pursuing it or in leaving it too late, or had I not looked at all of the strategic elements that I had to consider, then my operational plan for each week would have been ad hoc or non-existent. Worse, it may have been an unattainable plan; a waste of my time and the students’ efforts. We have far less excuse than George Lucas does for pretending that Star Wars was part of some enormous nine movie vision – although, to be fair, it doesn’t mean that this wasn’t somewhere in his head, but it obviously wasn’t sufficiently well plotted to guarantee a required level of consistency to make us really believe that statement.

The Big Picture is a framing that helps certain creative works drag you in and make more money, whereas in other words it is a valid structure that supports and develops consistency within a shared context. Our work as educators fits squarely into the final category. Without a solid plan, we risk making short-sighted decisions that please us or the student with ‘easy’ reward activities or the answers that come to hand at the time.

I’m not saying that certain elements have to be left out of our teaching, or that we have to be rigid in an inflexible structure, but consistency and reliability are two very important aspects of gaining student trust and, if holding it together over six serial instalments is too hard for Stephen King, then trying to achieve this, without some serious and detailed planning, over 36 lectures spanning four months is probably too much for most of us. The Big Picture, for us, is something that I believe we can find and use very effectively to make our teaching even better, effectively reducing our workload throughout the semester because we don’t have to carry out massive revisions or fixes, with a little more investment of time up front.

(Afterthought: I had no idea that Dr Steele has released an album called “Eclectic Boogaloo”. I was riffing on the old “Breakin’ 2: Electric Boogaloo” thing. In my defence, it was the 80s and we all looked like this:

This is considered one of the worst movies ever made. Yes, even when you take Gigli into account.

)

← Older Entries
Newer Entries →