The first paper, in the final session, was the “Effect of a 2-week Scratch Intervention in CS1 on Learners with Varying Prior Knowledge”, presented by Shitanshu Mirha, from IIT Bombay. The CS1 course context is a single programming course for all freshmen engineer students, thus it has to work for novice and advanced learners. It’s the usual problem: novices get daunted and advanced learners get bored. (We had this problem in the past.) The proposed solution is to use Scratch, because it’s low-floor (easy to get started), high-ceiling (can build complex projects) and wide-walls (applies to a wide variety of topics and themes). Thus it should work for both novice and advanced learners.
The theoretical underpinning is that novice learners reach cognitive overload while trying to learn techniques for programming and a language at the same time. One way to reduce cognitive load is to use visual programming environments such as Scratch. For advanced learners, Scratch can provide a sufficiently challenging set of learning material. From the perspective of Flow theory, students need to reach equilibrium between challenge level and perceived skill.
The research goal was to investigate the impact of a two-week intervention in a college course that will transition to C++. What would novices learn in terms of concepts and C++ transition? What would advanced students learn? What was the overall impact on students?
The cohort was 450 students, no CS majors, with a variety of advanced and novice learners, with a course objective of teaching programming in C++ across 14 weeks. The Scratch intervention took place over the first four weeks in terms of teaching and assessment. Novice scaffolding was achieved by ramping up over the teaching time. Engagement for advanced learners was achieved by starting the project early (second week). Students were assessed by quizzes, midterms and project production, with very high quality projects being demonstrated as Hall of Fame projects.
Students were also asked to generate questions on what they learned and these could be used for other students to practice with. A survey was given to determine student perception of usefulness of the Scratch approach.
The results for Novices were presented. While the Novices were able to catch up in basic Scratch comprehension (predict output and debug code), this didn’t translate into writing code in Scratch or debugging programs in C++. For question generation, Novices were comparable to advanced learners in terms of number of questions generated on sequences, conditionals and data. For threads, events and operators, Novices generated more questions – although I’m not sure I see the link that demonstrates that they definitely understood the material. Unsurprisingly, given the writing code results, Novices were weaker in loops and similar programming constructs. More than 53% of Novices though the Scratch framing was useful.
In terms of Advanced learner engagement, there were more Advanced projects generated. Unsurprisingly, Advanced projects were far more complicated. (I missed something about Most-Loved projects here. Clarification in the comments please!) I don’t really see how this measures engagement – it may just be measuring the greater experience.
Summarising, Scratch seemed to help Novices but not with actual coding or working with C++, but it was useful for basic concepts. The author claims that the larger complexity of Advanced user projects shows increased engagement but I don’t believe that they’ve presented enough here to show that. The sting in the tail is that the Scratch intervention did not help the Novices catch up to the Advanced users for the type of programming questions that they would see in the exam – hence, you really have to question its utility.
The next paper is “Enhancing Syntax Error Messages Appears Ineffectual” presented by Paul Denny, from The University of Auckland. Apparently we could only have one of Paul or Andrew Luxton-Reilly, so it would be churlish to say anything other than hooray for Paul! (Those in the room will understand this. Sorry we missed you, Andrew! Catch up soon.) Paul described this as the least impressive title in the conference but that’s just what science is sometimes.
Java is the teaching language at Auckland, about to switch to Python, which means no fancy IDEs like Scratch or Greenfoot. Paul started by discussing a Java statement with a syntax error in it, which gave two different (but equally unhelpful) error messages for the same error.
if (a < 0) || (a > 100) error=true; // The error is in the top line because there should be surrounding parentheses around conditions // One compiler will report that a ';' is required at the ||, which doesn't solve the right problem. // The other compiler says that another if statement is required at the || // Both of these are unhelpful - as well as being wrong. It wasn't what we intended.
The conclusion (given early) is simple: enhancing the error messages with a controlled empirical study found no significant effect. This work came from thinking about an early programming exercise that was quite straightforward but seemed to came students a lot of grief. For those who don’t know, programs won’t run until we fix the structural problems in how we put the program elements together: syntax errors have to be fixed before the program will run. Until the program runs, we get no useful feedback, just (often cryptic) error messages from the compiler. Students will give up if they don’t make progress in a reasonable interval and a lack of feedback is very disheartening.
The hypothesis was that providing more useful error messages for syntax errors would “help” users, help being hard to quantify. These messages should be:
- useful: simple language, informal language and targeting errors that are common in practice. Also providing example code to guide students.
- helpful: reduce the number of non-compiling submissions in total, reduce number of consecutive non-compiling submissions AND reduce the number of attempts to resolve a specific error.
In related work, Kummerfeld and Kay (ACE 2003), “The neglected battle fields of Syntax Errors”, provided a web-based reference guide to search for the error text and then get some examples. (These days, we’d probably call this Stack Overflow. 🙂 ) Flowers, Carver and Jackson, 2004, developed Gauntlet to provide more informal error messages with user-friendly feedback and humour. The paper was published in Frontiers in Education, 2004, “Empowering Students and Building Confidence in Novice Programmers Through Gauntlet.” The next aspect of related work was from Tom Schorsch, SIGCSE 1995, with CAP, making specific corrections in an environment. Warren Toomey modified BlueJ to change the error subsystem but there’s no apparent published work on this. The final two were Dy and Rodrigo, Koli Calling 2010, with a detector for non-literal Java errors and Debugging Tutor: Preliminary evaluation, by Carter and Blank, KCSC, January 2014.
The work done by the authors was in CodeWrite (written up in SIGCSE 2011 and ITiCSE 2011, both under Denny et al). All students submit non-compiling code frequently. Maybe better feedback will help and influence existing systems such as Nifty reflections (cloud bat) and CloudCoder. In the study, student had 10 problems they could choose from, with a method, description and return result. The students were split in an A/B test, where half saw raw feedback and half saw the enhanced message. The team built an error recogniser that analysed over 12,000 submissions with syntax errors from a 2012 course and the raw compiler message identified errors 78% of the time. (“All Syntax Errors are Not Equal”, ITiCSE 2012). In other cases, static analysis was used to work out what the error was. Eventually, 92% of the errors were classifiable from the 2012 dataset. Anything not in that group was shown as raw error message to the student.
In the randomised controlled experiment, 83 students had to complete the 10 exercises (worth 1% each), using the measures of:
- number of consecutive non-compiing submissions for each exercise
- Total number of non-compiling submissions
- … and others.
Do students even read the error messages? This would explain the lack of impact. However, examining student code change there appears to be a response to the error messages received, although this can be a slow and piecemeal approach. There was a difference between the groups, but it wasn’t significant, because there was a 17% reduction in non-compiling submissions.
I find this very interesting because the lack of significance is slightly unexpected, given that increased expressiveness and ease of reading should make it easier for people to find errors, especially with the provision of examples. I’m not sure that this is the last word on this (and I’m certainly not saying the authors are wrong because this work is very rigorous) but I wonder what we could be measuring to nail this one down into the coffin.
The final talk was “A Qualitative Think-Aloud Study of Novice Programmers’ Code Writing Strategies”, which was presented by Tony Clear, on behalf of the authors. The aim of the work was to move beyond the notion of levels of development and attempt to explore the process of learning, building on the notion of schemas and plans. Assimilation (using existing schemas to understand new information) and accommodation (new information won’t fit so we change our schema) are common themes in psychology of learning.
We’re really not sure how novice programmers construct new knowledge and we don’t fully understand the cognitive process. We do know that learning to program is often perceived as hard. (Shh, don’t tell anyone.) At early stages, movie programmers have very few schemas to draw on, their knowledge is fragile and the cognitive load is very high.
Woohoo, Vygotsky reference to the Zone of Proximal Development – there are things students know, things that can learn with help, and then the stuff beyond that. Perkins talked about attitudinal factors – movers, tinkerers and stoppers. Stoppers stop and give up in the face of difficulty, tinkers fiddle until it works and movers actually make good progress and know what’s going on. The final aspect of methodology was inductive theory construction, while I’ll let you look up.
Think-aloud protocol requires the student to clearly vocalise what they were thinking about as they completed computation tasks on a computer, using retrospective interviews to address those points in the videos where silence, incomprehensibility or confused articulation made interpreting the result impossible. The scaffolding involve tutoring, task performance and follow-up. The programming tasks were in a virtual world-based pogromming environment to solve tasks of increasing difficulty.
How did they progress? Jacquie uses the term redirection to mean that the student has been directed to re-examine their work, but is not given any additional information. They’re just asked to reconsider what they’ve done. Some students may need a spur and then they’re fine. We saw some examples of students showing their different progression through the course.
Jacquie has added a new category, PLANNERS, which indicates that we can go beyond the Movers to explain the kind of behaviour we see in advanced students in the top quartile. Movers who stretch themselves can become planners if they can make it into the Zone of Proximal Development and, with assistance, develop their knowledge beyond what they’d be capable of by themselves. The More Competent Other plays a significant role in helping people to move up to the next level.
Full marks to Tony. Presenting someone else’s work is very challenging and you’d have to be a seasoned traveller to even reasonably consider it! (It was very nice to see the lead author recognising that in the final slide!)
CSEDU, Day 3, Final Keynote, “Digital Age Learning – The Changing Face of Online Education”, (#csedu14 #AdelED @timbuckteeth)Posted: April 4, 2014
Now, I should warn you all that I’ve been spending time with Steve Wheeler (@timbuckteeth) and we agree on many things, so I’m either going to be in furious agreement with him or I will be in shock because he suddenly reveals himself to be a stern traditionalist who thinks blended learning is putting a textbook in the Magimix. Only time will tell, dear reader, so let’s crack on, shall we? Steve is from the Plymouth Institute of Education, conveniently located in Plymouth University, and is a ferocious blogger and tweeter (see his handle above).
Erik introduced Steve by saying that Steve didn’t need much introduction and noted that Steve was probably one of the reasons that we had so many people here on the last day! (This is probably true, the afternoon on the last day of a European conference is normally notable due to the almost negative number of participants.)
“When you’re a distance educator, the back of the classroom can be thousands of miles away” (Steve Wheeler)
Steve started with the idea that on-line learning is changing and that his presentation was going to be based on the idea that the future will be richly social and intensely personal. Paradoxical? Possibly but let’s find out. Oh, look, an Einstein quote – we should have had Einstein bingo cards. It’s a good one and it came with an anecdote (which was a little Upstairs Downstairs) so I shall reproduce it here.
“I never teach my students. I only provide the conditions in which they can learn.” Albert Einstein
There are two types of learning: shallow (rote) learning that we see when cramming, where understanding is negligible or shallow if there at all, and then there is the fluid intelligence, the deeper kind of learning that draws on your previous learning and your knowledge structures. But what about strategic learning where we switch quickly between the two. Poor pedagogy can suppress these transitions and lock people into one spot.
There are three approaches here: knowledge (knowing that, which is declarative), wisdom (knowing how, which is procedural) and transformation (knowing why, which is critical). I’ve written whole papers about the missing critical layer so I’m very happy to see Steve saying that the critical layer is the one that we often do the worst with. This ties back into blooms where knowledge is cognitive, wisdom is application and transformation is analysis and evaluation. Learning can be messy but it’s transformative and it can be intrinsically hard to define. Learning is many things – sorry, Steve, not going to summarise that whole sentence.
We want to move through to the transformational stage of learning.
What is the first attempt at distance learning? St Paul’s name was tossed out, as was Moses. But St Paul was noted as the first correspondence course offered. (What was the assessment model, I wonder, for Epistola.) More seriously, it was highly didactic and one-way, and it was Pitman who established a two-way correspondence course that was both laborious and asynchronous but it worked. Then we had television and in 1968, the Stanford Instructions TV Network popped up. In 1970, Steve saw an example of video conferencing that had been previously confined to Star Trek. I was around in the early 70s and we were all agog about the potential of the future – where is my moon base, by the way? But the tools were big and bulk – old video cameras were incredibly big and ridiculously short lived in their battery life… but it worked! Then people saw uses for the relationship between this new technology and pedagogy. Reel-to-reel, copiers, projectors, videos: all of these technologies were effective for their teaching uses at the time.
Of course, we moved on to computer technology including the BBC Model B (hooray!) and the reliable but hellishly noisy dot matrix printer. The learning from these systems was very instructional, using text and very simplistic in multiple choice question approach. Highly behaviouristic but this is how things were done and the teaching approach matched the technology. Now, of course, we’ve gone tablet-based, on-line gaming environments that have non-touch technologies such as Kinect, but the principle remains the same: over the years we’ve adapted technology to pedagogy.
But it’s only now that, after Sir Tim Berners-Lee, we have the World Wide Web that on-line learning is now available to everybody, where before it was sort-of available but not anywhere near as multiplicable. Now, for our sins, we have Learning Management Systems, the most mixed of blessings, and we still have to ask what are we using them for, how are we using them? Is our pedagogy changing? Is out connection with our students changing? Illich (1972) criticised educational funnels that had a one-directional approach and intend motivated educational webs that allow the transformation of each moment of living into one of learning, sharing and caring.
What about the Personal Learning Environment (PLE)? This is the interaction of tools such as blogs, twitters and e-Portfolios, then add in the people we interact with, and then the other tools that we use – and this would be strongly personal to an individual. If you’ve ever tried to use your partner’s iPad, you know how quickly personalisation changes your perception of a tool! Wheeler and Malik (2010) discuses the PLE that comprises the personal learning network and personal web tools, with an eye on more than the classroom, but as a part of life-long learning. Steve notes (as Stephen Heppel did) that you may as well get students to use their PLEs in the open because they’ll be using them covertly otherwise: the dreaded phone under the table becomes a learning tool when it’s on top of the table. Steve discussed the embedded MOOC that Hugh discussed yesterday to see how the interaction between on-line and f2f students can benefit from each other.
In the late ’80s, the future was “multi-media” and everything had every other medium jammed into it (and they don’t like it up ’em) and then the future was going to converge on the web. Internet take up is increasing: social, political and economic systems change incrementally, but technology changes exponentially. Steve thinks the future is smart mobile and pervasive, due to miniaturisation and capability of new devices. If you have WiFi then you have the world.
“Change is not linear, it’s exponential.” Kurzweil
Looking at the data, there are no more people in the world with mobile phones than people without, although some people have more than one. (Someone in the audience had four, perhaps he was a Telco?) Of course, some reasons for this are because mobile phones replace infrastructure: there are entire African banks that run over mobile networks, as an example. Given that we always have a computer in our pocket, how can we promote learning everywhere? We are using these all the time, everywhere, and this changes what we can do because we can mix leisure and learning without having to move to fixed spaces.
Steve then displayed the Intel info graphic “What Happens In an Internet Minute“, but it’s scary to see how much paper is lagging these days. What will the future look like? What will future learning look like? If we think exponentially then things are changing fast. There is so much content being generated, there must be something that we can use (DOGE photos and Justin Bieber vides excepted) for our teaching and learning. But, given that 70% of what we learn is if informal and outside of the institution, this is great! But we need to be able to capture this and this means that we should produce a personal learning network, because trying to drink down all that content by yourself is exceeding our ability! By building a network, we build a collection of filters and aggregators that are going to help us to bring sense out of the chaos. Given that nobody can learn everything, we can store our knowledge in other people and know where to go when we need that knowledge. A plank of connectivist theory and leading into paragogy, where we learn from each other. This also leads us to distributed cognition, where we think across the group (a hive mind, if you will) but, more simply, you learn from one person, then another, and it becomes highly social.
Steve showed us a video on “How have you used your own technology to enhance your learning“, which you can watch on YouTube. Lucky old 21st Century you! This is a recording of some of Steve’s students answering the question and sharing their personal learning networks with us. There’s an interesting range of ideas and technologies in use so it’s well worth a look. Steve runs a Twitter wall in his classroom and advertises the hashtag for a given session so questions, challenges and comments go out on to that board and that allows Steve to see it but also retweet it to his followers, to allow the exponential explosion that we would want in a personal learning network. Students accessed when they harness the tools they need to solve their problems.
Steve showed us a picture of about 10,000 Germans taking pictures of the then-Presidential Elect Barack Obama because he was speaking in Berlin and it was a historical moment that people wanted to share with other people. This is an example of the ubiquitous connection that we now enjoy and, in many ways, take for granted. It is a new way of thinking and it causes a lot of concern for people who want to stick to previous methods. (There will come a time when a paper exam for memorised definitions will make no sense because people have computers connected to their eyes – so let’s look at asking questions in ways that always require people to actually use their brains, shall we.) Steve then showed us a picture of students “taking notes” by taking pictures of the whiteboard: something that we are all very accustomed to now. Yes, some teachers are bothered by this but why? What is wrong with instantaneous capture versus turning a student into a slow organic photocopying machine? Let’s go to a Papert quote!
“I am convinced that heh best learning takes place when the learner takes charge,” Seymour Papert
“We learn by doing“, Piaget, 1960
“We learn by making“, Papert, 1960.
Steve alluded to constructionist theory and pointed out how much we have to learn about learning by making. He, like many of us, doesn’t subscribe to generational or digital native/immigrant theory. It’s an easy way of thinking but it really gets in the way, especially when it makes teachers fearful of weighing in because they feel that their students know more than they do. Yes, they might, but there is no grand generational guarantee. It’s not about your age, it’s about your context. It’s about how we use the technology, it’s not about who we are and some immutable characteristics that define us as in or out. (WTF does not, for the record, mean “Welcome to Facebook”. Sorry, people.) There will be cultural differences but we are, very much, all in this together.
Steve showed us a second video, on the Future of Publishing, which you can watch again! Some of you will find it confronting that Gaga beats Gandhi but cultures change and evolve – and you need to watch to the end of the video because it’s really rather clever. Don’t stop halfway through! As Steve notes, it’s about perception and, as I’ve noted before, I’m pretty sure that people put people into the categories that they were already thinking about – it’s one of the reasons I have such a strong interest in grounded theory. If you have a “Young bad” idea in your head then everything you see will tend to confirm this. Perception and preconception can heavily interfere with each other but using perception, and being open to change, is almost always a better idea.
Steve talked about Csíkszentmihályi’s Flow, the zone you’re in when the level of challenge roughly matches your level of skill and you balance anxiety and boredom. Then, for maximum Nick points, he got onto Vygotsky’s Zone of Proximal Development, where we build knowledge better and make leaps when we do it with other people, using the knowledgable other to scaffold the learning. Steve also talked about mashing them up, and I draw the reader back to something I wrote on this a whole ago on Repenning’s work.
We can do a lot of things with computers but we don’t have to do all the things that we used to do and slavishly translate them across to the new platform. Waters (2011) talks about new learners: learners who are more self-directed and able to make more and hence learn more.
There are many digital literacies: social networking, privacy management, identity management, creating content, organising content, reusing and repurposing, filtering and selection, self presentation, transliteracy (using any platform to get your ideas across). We build skills, that become competencies, that become literacies and, finally, potentially become masteries.
Steve finished with in discussing the transportability of skills using driving in the UK and the US as an example. The skill is pretty much the same but safe driving requires a new literacy when you make a large contextual change. Digital environments can be alien environments so you need to be able to take the skills that you have now and be able to put them into the new contexts. How do you know that THIS IS SHOUTING? It’s a digital literacy.
Steve presented a quote from Socrates, no, Socrates, no, Plato:
“Knowledge that is acquired under compulsion obtains no hold on the mind.“
and used the rather delightful neologism “Darwikianism” to illustrate evolving improvement on on-line materials over time. (And illustrated it with humour and pictures.) Great talk with a lot of content! Now I have to go and work on my personal learning network!
While I’ve been talking about the project work in my new “Grand Challenge”-based course a lot, I’ve also identified a degree of, for want of a better word, fearfulness on the part of the students. Given that their first project is a large poster with a visualisation of some interesting data, which they have to locate and analyse, and that these are mostly Computer Science students with no visualisation experience, they are understandably slightly concerned. We’ve been having great discussions and lots of contributions but next week is their first pitch and, suddenly, they need a project theme.
I’ve provided a fair bit of guidance for the project pitch, and I reproduce it here in case you’re interested:
Project 1: First Deliverable, the Pitch
Due 2pm, Wednesday, the 8th of August Because group feedback is such an important part of this project, you must have your pitch ready to present for this session and have the best pitch ready that you can. Allocate at least 10 hours to give you enough time to do a good job.
What is the pitch?
A pitch is generally an introduction of a product or service to an audience who knows nothing about it but is often used to expand knowledge and provide a detailed description of something that the audience is already partially familiar with. The key idea is that you wish to engage your audience and convince them that what you are proposing is worth pursuing. In film-making, it’s used to convey an idea to people who need to agree to support it from a financial or authority perspective.
One of the most successful pitches in Hollywood history is (reputedly) the four word pitch used to convince a studio to fund the movie “Twins”. The pitch was “Schwarzenegger. De Vito. Twins.”
You are not trying to sell anything but you are trying to familiarise a group of people with your project idea and communicate enough information that the group can give you useful feedback to improve your project. You need to think carefully about how you will do this and I strongly suggest that you rehearse before presenting. Trust me when I say that very few people are any good at presentation without rehearsal and I will generally be able to tell the amount of effort that you’ve expended. An indifferent presentation says that you don’t care – and then you have to ask why anyone else would be that motivated to help you.
If you like the way I lecture, then you should know that I still rehearse and practice regularly, despite having been teaching for over 20 years.
How will it work?
You will have 10 minutes to present your project outline. During this time you will:
- Identify, in one short and concise sentence, what your poster is about.
- Clearly state the purpose.
- Identify your data source.
- Answer all of the key questions raised in the tutorial.
- Identify your starting strategy, based on the tools given in the tutorial, with a rough outline of a timeline.
- Outline your analysis methodology.
- Summarise the benefits of this selection of data and presentation – why is it important/useful?
- Show a rough prototype layout on an A3 format.
We will then take up to 10 minutes to provide you with constructive feedback regarding any of these aspects. Participants will be assessed both on the pitch that they present and the quality of their feedback and critique. Critique guides will be available for this session.
How do I present it?
This is up to you but I would suggest that you summarise the first seven points as a handout, and provide a copy of your A3 sketch, for reference during critique. You may also use presentations (PowerPoint, Keynote or PDF) if you wish, or the whiteboard. As a guideline, I would suggest no more than four slides, not including title, or your poster sketch. You may use paper and just sketch on that – the idea and your ability to communicate it are paramount at this stage, not the artfulness of the rough sketch.
Some people haven’t been getting all of their work ready on time and, up until now, this has had no impact on your marks or your ability to continue working with the group. If you don’t have your project ready, then I cannot give you any marks for your project and you miss out on the opportunity for group critique and response – this will significantly reduce your maximum possible mark for this project.
I am interested in you presenting something that you find interesting or that you feel will benefit from working with – or that you think is important. The entire point of this course is to give you the chance to do something that is genuinely interesting and to challenge yourself. Please think carefully about your data and your approach and make sure that you give yourself the opportunity to make something that you’d be happy to show other people, as a reflection of yourself, your work and what you are capable of.
END OF THE PITCH DESCRIPTION
We then had a session where we discussed ideas, looked at sources and started to think about how we could get some ideas to build a pitch on. I used small group formation and a bit of role switching and, completely unsurprisingly to the rest of you social constructivists, not only did we gain benefit from the group work but it started to head towards a self-sustaining activity. We went from “I’m not really sure what to do” to something very close to “flow” for the majority of the class. To me it was obvious that the major benefit was that the ice had been broken and, through careful identification of what to happen with the ideas and a deliberate use of Snow’s Cholera diagram as an example of how powerful a good (but fundamentally) simple visualisation could be, the group was much better primed to work on the activity.
The acid test will be next week but, right now, I’m a lot more confident that I will get a good set of first pitches. Given how much I was holding my breath, without realising it, that’s quite a good thing!
I’ve just been reading Deirdre McCloskey’s article on “Happyism” in The New Republic. While there are a number of points I could pick at in the article, I question her specific example of statistical significance and I think she’s oversimplified a number of the philosophical points, there are a lot of interesting thoughts and arguments within the article.
One of my challenges in connecting with my students is that of making them understand what the benefit is to them of adopting, or accepting, suggestions from me as to how to become better as discipline practitioners, as students and, to some extent, as people. It would be nice if doing the right thing in this regard could give the students a tangible and measurable benefit that they could accumulate on some sort of meter – I have performed well, my “success” meter has gone up by three units. As McCloskey points out, this effectively requires us to have a meter for something that we could call happiness, but it is then tied directly to events that give us pleasure, rather than a sequence of events that could give us happiness. Workflows (chains of actions that lead to an eventual outcome) can be assessed for accuracy and then the outcome measured, but it is only when the workflow is complete that we can assess the ‘success’ of the workflow and then derive pleasure, and hence happiness, from the completion of the workflow. Yes, we can compose a workflow from sub-workflows but we will hit the same problem if we focus on an outcome-based model – at some stage, we are likely to be carrying out an action that can lead to an event from which we can derive a notion of success, but this requires us to be foresighted and see the events as a chain that results in this outcome.
And this is very hard to meter and display in a way that says anything other than “Keep going!” Unsurprisingly, this is not really the best way to provide useful feedback, reward or fodder for self-actualisation.
I have a standing joke that, as a runner, I go to a sports doctor because if I go to a General Practitioner and say “My leg hurts after I run”, the GP will just say “Stop running.” I am enough of a doctor to say that to myself – so I seek someone who is trained to deal with my specific problems and who can give me a range of feedback that may include “stop running” because my injuries are serious or chronic, but can provide me with far more useful information from which I can make an informed choice. The happiness meter must be able to work with workflow in some way that is useful – keep going is not enough. We therefore need to look at the happiness meter.
McCloskey identifies Bentham, founder of utilitarianism, as the original “pleasure meter” proponent and implicitly addressed the beneficial calculus as subverting our assessment of “happiness units” (utils) into a form that assumes that we can reasonably compare utils between different people and that we can assemble all of our life’s experiences in a meaningful way in terms of utils in the first place!
To address the issue of workflow itself, McCloskey refers to the work of Mihály Csíkszentmihályi on flow: “the absorption in a task just within our competence”. I have talked about this before, in terms of Vygotsky’s zone of proximal development and the use of a group to assist people who are just outside of the zone of flow. The string of activities can now be measured in terms of satisfaction or immersion, as well as the outcomes of this process. Of course, we have the outcomes of the process in terms of direct products and we have outcomes in terms of personal achievement at producing those products. Which of these go onto the until meter, given that they are utterly self-assessed, subjective and, arguably, orthogonal in some cases. (If you have ever done your best, been proud of what you did, but failed in your objective, you know what I’m talking about.)
My reading of McCloskey is probably a little generous because I find her overall argument appealing. I believe that her argument may be distilled are:
- If we are going to measure, we must measure sensibly and be very clear in our context and the interpretation of significance.
- If we are going to base any activity on our measurement, then the activity we create or change must be related to the field of measurement.
Looking at the student experience in this light, asking students if they are happy with something is, ultimately, a pointless activity unless I either provide well-defined training in my measurement system and scale, or I am looking for a measurement of better or worse. This is confounded by simple cognitive biasses including, but not limited to, the Hawthorne Effect and confirmation bias. However, measuring what my students are doing, as Csíkszentmihályi did in the flow experiments, will show me if they are so engaged with their activities that they are staying in the flow zone. Similarly, looking at participation and measuring outputs in collaborative activities where I would expect the zone of proximal development to be in effect is going to be far more revealing than asking students if they liked something or not.
As McCloskey discusses, there is a point at which we don’t seem to get any happier but it is very hard to tell if this is a fault in our measurement and our presumption of a three-point non-interval scale and it then often degenerates into a form of intellectual snobbery that, unsurprisingly, favours the elites who will be studying the non-elites. (As an aside, I learnt a new word. Clerisy: “A distinct class of learned or literary people” If you’re going to talk about the literate elites, it’s nice to have a single word to do so!) In student terms, does this mean that there is a point at which even the most keen of our best and brightest will not try some of our new approaches? The question, of course, is whether the pursuit of happiness is paralleling the quest for knowledge, or whether this is all one long endured workflow that results in a pleasure quantum labelled ‘graduation’.
As I said, I found it to be an interesting and thoughtful piece, despite some problems and I recommend it to you, even if we must then start an large debate in the comments on how much I misled you!
One of my colleagues drew my attention to an article in a recent Communications of the ACM, May 2012, vol 55, no 5, (Education: “Programming Goes to School” by Alexander Repenning) discussing how we can broaden participation of women and minorities in CS by integrating game design into middle school curricula (Thanks, Jocelyn!). The article itself is really interesting because it draws on a number of important theories in education and CS education but puts it together with a strong practical framework.
There’s a great diagram in it that shows Challenge versus Skills, and clearly illustrates that if you don’t get the challenge high enough, you get boredom. Set it too high, you get anxiety. In between the two, you have Flow (from Csíkszentmihályi’ s definition, where this indicates being fully immersed, feeling involved and successful) and the zone of proximal development (ZPD).
Which brings me to Vygotsky. Vgotsky’s conceptualisation of the zone of proximal development is designed to capture that continuum between the things that a learner can do with help, and the things that a learner can do without help. Looking at the diagram above, we can now see how learners can move from bored (when their skills exceed their challenges) into the Flow zone (where everything is in balance) but are can easily move into a space where they will need some help.
Most importantly, if we move upwards and out of the ZPD by increasing the challenge too soon, we reach the point where students start to realise that they are well beyond their comfort zone. What I like about the diagram above is that transition arrow from A to B that indicates the increase of skill and challenge that naturally traverses the ZPD but under control and in the expectation that we will return to the Flow zone again. Look at the red arrows – if we wait too long to give challenge on top of a dry skills base, the students get bored. It’s a nice way of putting together the knowledge that most of us already have – let’s do cool things sooner!
That’s one of the key aspects of educational activities – not they are all described in terms educational psychology but they show clear evidence of good design, with the clear vision of keeping students in an acceptably tolerable zone, even as we ramp up the challenges.
One the key quotes from the paper is:
The ability to create a playable game is essential if students are to reach a profound, personally changing “Wow, I can do this” realization.
If we’re looking to make our students think “I can do this”, then it’s essential to avoid the zone of anxiety where their positivity collapses under the weight of “I have no idea how anyone can even begin to help me to do this.” I like this short article and I really like the diagram – because it makes it very clear when we overburden with challenge, rather than building up skill and challenge in a matched way.