De Profundis – or de-profounding?

Posted: September 16, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , , , | Leave a comment

“It is common to assume that we are dealing with a highly intelligent book when we cease to understand it.” (de Botton, The Consolations of Philosophy, p157)

The notion of a lack of comprehension being a fundamental and innate fault of the reader, rather than the writer, is a mistake made, in many different and yet equally irritating ways, throughout the higher educational sector. A high pass rate may be seen as indicative of an easy course or a weak marker. A high failure rate may be attributed to the innate difficulty of the work or the inferior stuff of which the students are made. As I have written before, under such a presumption, I could fail all of my students and strut around, the smartest man in my University, for none have been able to understand the depths and subtlety of my area of knowledge.

Yet, if the real reason is that I have brought my students to a point where their abilities fail them and, either through ignorance or design, I do not strive to address this honestly and openly, then it doesn’t matter how many of them ultimately pass – I will be the biggest failure in the class. I know a great number of very interesting and intelligent educators but, were you to ask me if any of them could teach, I would have to answer that I did not know, unless I had actually seen them do so. For all of our pressure on students to contain the innate ability to persevere, to understand our discipline or to be (sorry, Ray) natural programmers, the notion that teaching itself might not be something that everyone is capable of is sometimes regarded as a great heresy. (The notion or insistence that developing as a teacher may require scholarship and, help us all, practise, is apostasy – our heresy leading us into exile.) Teaching revolves around imparting knowledge efficiently and effectively so that students may learn. The cornerstone of this activity is successful and continuing communication. Wisdom may be wisdom but it rapidly becomes hard to locate or learn from when it is swaddled in enough unnecessary baggage.

I have been, mostly thanks to the re-issue of cheap Penguins, undertaking a great deal of reading recently and I have revisited Marcus Aurelius, Seneca, de Botton and Wilde. The books that are the most influential upon me remain those books that, while profound, maintain their accessibility. Let me illustrate this with an example. For those who do not know what De Profundis means, it is a biblical reference to Psalm 130, appropriated by the ever humble Oscar Wilde as the title of his autobiographical letter to his former lover, from the prison in which he was housed because of that love.

But what it means is “From the depths”. In the original psalm, the first line is:

De profundis clamavi ad te, Domine;
From the depths, I have cried out to you, O Lord;

And in this reading, we see the measure of Wilde’s despair. Having been sentenced to hard labour, and having had his ability to write confiscated, his ability to read curtailed, and his reputation in tatters, he cries out from the depths to his Bosie, Lord Douglas.

De profundis [clamavi ad te, Bosie;]

If you have the context for this, then this immediately prepares you for the letter but, as it is, the number of people who are reading Wilde is shrinking, let alone the number of people who are reading a Latin Bible. Does this title still assist in the framing of the work, through its heavy dependence upon the anguish captured in Psalm 130, or is it time to retitle it “From the depths, I have cried out to you!” to capture both the translation and the sense. The message, the emotion and the hard-earned wisdom contained in the letter are still valuable but are we hurting the ability of people to discover and enjoy it by continuing to use a form of expression that may harm understanding?

Les Très Riches Heures du duc de Berry, Folio 70r – De Profundis the Musée Condé, Chantilly. (Another form of expression of this Psalm.)

Now, don’t worry, I’m not planning to rewrite Wilde but this raises a point in terms of the occasionally unhappy union of the language of profundity and the wisdom that it seeks to impart. You will note the irony that I am using a heavily structured, formal English, to write this and that there is very little use of slang here. This is deliberate because I am trying to be precise while still being evocative and, at the same time, illustrating that accurate use of more ornate language can obscure one’s point. (Let me rephrase that. The unnecessary use of long words and complex grammar gets in the way of understanding.)

When Her Majesty the Queen told the Commonwealth of her terrible year, her words were:

“1992 is not a year on which I shall look back with undiluted pleasure. In the words of one of my more sympathetic correspondents, it has turned out to be an Annus Horribilis.”

and I have difficulty thinking of a more complicated way of saying “1992 was a bad year” than to combine a complicated grammatical construction with a Latin term that is not going to be on the lips of the people who are listening to the speech. Let me try: “Looking back on 1992, it has been, in the words of one of my friends, a terrible year.” Same content. Same level of imparted knowledge. Much less getting in the way. (The professional tip here is to never use the letters “a”, “n”, “s” and “u” in one short word unless you are absolutely sure of your audience. “What did she say about… nahhh” is not the response you want from your loyal subjects.) [And there goes the Knighthood.]

I love language. I love reading. I am very lucky that, having had a very broad and classically based education, I can read just about anything and not be intimidated or confused by the language forms – providing that the author is writing in one of the languages that I read, of course! To assume that everyone is like me or, worse, to judge people on their ability because they find long and unfamiliar words confusing, or have never had the opportunity to use these skills before, is to leap towards the same problem outlined in the quote at the top. If we seek to label people unintelligent when they have not yet been exposed to something that is familiar to us, then this is just as bad as lauding someone’s intelligence because you don’t understand what they’re talking about.

If my students need to know something then I have to either ensure that they already do so, by clearly stating my need and being aware of the educational preparation in my locale, or I have to teach it to them in forms that they can understand and that will allow them to succeed. I may love language, classical works and big words, but I am paid to teach the students of 2012 to become the graduates, achievers and academics of the future. I have to understand, respect and incorporate their context, while also meeting the pedagogical and knowledge requirements of the courses that I teach.

No-one said it was going to be easy!

ICER 2012 Day 3 Research Paper Session 5

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , | Leave a comment

The last of the research paper sessions and, dear reader, I am sure that you are as glad as I that we are here. Reading about an interesting conference that you didn’t attend is a bit like receiving a message from a friend talking about how he kissed the person that you always loved from afar. Thanks for the information but I would rather have been there myself.

This session opened with “Toward a Validated Computing Attitudes Survey” (Allison Elliott Tew, Brian Dorn and Oliver Schneider), where the problems with negative perceptions of the field and hostile classroom environments, combined with people thinking that they would be no good at CS, conspire to prevent students coming in to, or selecting, our discipline. The Computing Attitudes Survey was built, with major modification, from the Colorado Learning Attitudes about Science Survey (CLASS, pronounced C-LASS). To adapt the original survey, some material was just copied across with a word change (computer science replacing physics), some terminology was changed (algorithm for formula) and some discipline specific statements were added. Having established an expert opinion basis for the discipline specific content, students can now see how much they agree with the experts.

There is, as always, the rip of contentious issues. “You have to know maths to be able to program” was a three-way split within the expert group as to who agreed, disagreed or was neutral. What was interesting, and what I’ll be looking at in future, is the evidence of self-defeating thought in many answers (no, not questions. The questions weren’t self-defeatist but the answers often were.) What was also interesting is that attitudes seem to get worse in the CLASS instrument after you take the course!

Confidence, as simple as “I think I can do this”, plays a fundamental part in determining how students will act. Given the incredibly difficult decisions that a student faces when selecting their degree or concentration, it is no surprise that anyone who thinks “Computing is too hard for me” or “Computing is no use to me” will choose to do something else.

The authors are looking for volunteers where they can run these trials again so, after you’ve read their paper, if you’re interested, you should probably e-mail them.

“A Statewide Survey on Computing Education Pathways and Influences: Factors in Broadening Participation in Computing” (Mark Guzdial, Barbara Ericson, Tom McKlin and Shelly Engelman)

The final research paper in the conference dealt with the final evaluation of the Georgia Computes! initiative, which had run from October 2006 to August of this year. This multi-year project cannot be contained in my nervous babbling but I can talk about the instrument that was presented. Having run summer camps, weekend workshops, competitions, teacher workshops, a teachers’ lending library, first year engagement and seeded first-year summer camps (whew!), the question was: What had been the impact of Georgia Computes! ? What factors influence undergrad enrolment into intro CS courses?

There were many questions and results presented but I’d like to focus on the top four reasons given, from survey, as to why students weren’t going to undertake a CS Major or Minor:

  1. I don’t want to do the type of work
  2. Little interest in the subject matter
  3. Don’t enjoy Computing Courses
  4. Don’t have confidence that I would succeed.

Looking at those points, after a state-wide and highly successful campaign over 6 years has finished, it is very, very sobering for me. What these students are saying is that they cannot see the field as attractive, interesting, enjoyable or that they are capable. But these are all aspects that we can work on, although some of these will require a lot of work.

Two further things that Barb said really struck me. Firstly, that if you take into account encouragement and ability, that men will tend to be satisfied and continue on if they receive either or both – the factors are not separable for men – but that women and minorities need encouragement in order to feel satisfied and to convince them to keep going. Secondly, when it comes to giving encouragement, male professors are just as effective as female professors in terms of giving encouragement to women.

As a male lecturer, who is very, very clearly aware of the demographic disgrace that is the under-representation of women in CS, this first fact gives me a partial strategy to increase retention (and reinforces a believe I have held anecdotally for some time) but the second fact gives me the agency to assist in this process, as well as greater hope for a steadily increasing female cohort over time.

Overall, a very positive note on which to finish the session papers!

ICER 2012 Day Research Paper Session 4

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , | Leave a comment

This session kicked off with “Ability to ‘Explain in Plain English’ Linked to Proficiency in Computer-based Programming”,  (Laurie Murphy, Sue Fitzgerald, Raymond Lister and Renee McCauley (presenting)). I had seen a presentation along these lines at SIGCSE and this is an excellent example of international collaboration, if you look at the authors list. Does the ability to explain code in plain English correlate with ability to solve programming problems? The correlation appears to be there, whether or not we train students in Explaining in Plain English or not, but is this causation?

This raises a core question, addressed in the talk: Do we need to learn to read (trace) code before we learn to write code or vice versa? The early reaction of the Leeds group was that reading code didn’t amount to testing whether students could actually write code. Is there some unknown factor that must be achieved before either or both of these? This is a vexing question as it raises the spectre of whether we need to factor in some measure of general intelligence, which has not been used as a moderating factor.

Worse, we now return to that dreadful hypothesis of “programming as an innate characteristics”, where you were either born to program or not. Ray (unsurprisingly) believes that all of the skills in this area (EIPE/programming) are not innate and can be taught. This then raises the question of what the most pedagogically efficient way is to do this!

How Do Students Solve Parsons Programming Problems? — An Analysis of Interaction Traces
Juha Helminen (presenting), Petri Ihantola, Ville Karavirta and Lauri Malmi

This presentation was of particular interest to me because I am currently tearing apart my 1,900 student data corpus to try and determine the point at which students will give up on an activity, in terms of mark benefit, time expended and some other factors. This talk, which looked at how students solved problems, also recorded the steps and efforts that they took in order to try and solve them, which gave me some very interesting insights.

A Parsons problem is one where, given code fragments a student selects, arranges and composes a program in response to a question. Not all code fragments present will be required in the final solution. Adding to the difficulty, the fragments require different indentation to assert their execution order as part of block structure. For those whose eyes just glazed over, this means that it’s more than selecting a line to go somewhere, you have to associate it explicitly with other lines as a group. Juha presented a graph-based representation of the students’ traversals of the possible solutions for their Parsons problem. Students could ask for feedback immediately to find out how their programs were working and, unsurprisingly, some opted for a lot of “Am I there yet” querying. Some students queried feedback as much as 62 times for only 7 features, indicative of permutation programming, with very short inter-query intervals. (Are we there yet? No. Are we there yet? No. Are we there yet? No. Are we there yet? No.)

The primary code pattern of development was linear, with block structures forming the first development stages, but there were a lot of variations. Cycles (returning to the same point) also occurred in the development cycle but it was hard to tell if this was a deliberate reset pattern or a one where permutation programming had accidentally returned the programmer to the same state. (Asking the students “WHY” this had occurred would be an interesting survey question.)

There were some good comments from the audience, including the suggestion of correlating good and bad states with good and bad outcomes, using Markov chain analysis to look for patterns. Another improvement suggested was recording the time taken for the first move, to record the impact (possible impact) of cognition on the process. Were students starting from a ‘trial and error’ approach or only after things went wrong?

Tracking Program State: A Key Challenge in Learning to Program
Colleen Lewis (presenting, although you probably could have guessed that)

This paper won the Chairs’ Award for the best paper at the Conference and it was easy to see why. Colleen presented a beautifully explored case study of an 11 year old boy working on a problem in the Scratch programming language and trying to work out why he couldn’t draw a wall of bricks. By capturing Kevin’s actions, in code, his thoughts, from his spoken comments, we are exposed to the thought processes of a high achieving young man who cannot fathom why something isn’t working.

I cannot do justice to this talk by writing about something that was primarily visual, but Colleen’s hypothesis was that Kevin’s attention to the state (variables and environmental settings over which the program acts) within the problem is the determining factor in the debugging process. Once Kevin’s attention was focused on the correct problem, he solved it very quickly because the problem was easy to solve. Locating the correct problem required him to work through and determine which part of the state was at fault.

Kevin has a pile of ideas in his head but, as put by duSessa and Sherin (98), learning is about reliably using the right ideas in the correct context. Which of Kevin’s ideas are being used correctly at any one time? The discussion that followed talked about a lot of the problems that students have with computers, in that many students do not see computers as actually being deterministic. Many students, on encountering a problem, will try exactly the same thing again to see if the error occurs again – this requires a mental model that we expect a different set of outcomes with the same inputs and process, which is a loose definition of either insanity or nondeterminism. (Possibly both.)

I greatly enjoyed this session but the final exemplar, taking apart a short but incredibly semantically rich sequence and presenting it with a very good eye for detail, made it unsurprising that this paper won the award. Congratulations again, Colleen!

The Narrative Hunger: Stories That Meet a Need

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , | Leave a comment

I have been involved in on-line communities for over 20 years now and, apparently, people are rarely surprised when they meet me. “Oh, you talk just like you type.” is the effective statement and I’m quite happy with this. While some people adopt completely different personae on-line, for a range of reasons, I seem to be the same. It then comes as little surprise that I am as much of storyteller in person as I am online. I love facts, revel in truth, but I greatly enjoying putting them together into a narrative that conveys the information in a way that is neither dry nor dull. (This is not to say that the absence of a story guarantees that things must be dry and dull but, without a focus on those elements of narrative that appeal to common human experience, we always risk this outcome.)

One of Katrina’s recent posts referred to the use of story telling in education. As she says, this can be contentious because:

stories can be used to entertain students, to have them enjoy your lectures, but are not necessarily educational.

The shibboleth of questionable educational research is often a vaguely assembled study, supported by the conjecture that the “students loved it”, and it is very easy to see how story telling could fall into this. However, we as humans are fascinated by stories. We understand the common forms even where we have not read Greek drama or “The Hero With a Thousand Faces”. We know when stories ring true and when they fall flat. Searching the mental engines of our species for the sweet spots that resonate across all of us is one way to convey knowledge in a more effective and memorable way. Starting from this focus, we must then observe our due diligence in making sure that our story framework contains a worthy payload.

Not all stories are of the same value.

I love story telling and I try to weave together a narrative in most of my lectures, even down to leaving in sections where deliberate and tangential diversion becomes part of the teaching, to allow me to contrast a point or illuminate it further by stripping it of its formal context and placing it elsewhere. After all, an elephant next to elephants is hardly memorable but an elephant in a green suit, as King of a country, tends to stick in the mind.

The power of the narrative is that it involves the reader or listener in the story. A well-constructed narrative leads the reader to wonder about what is going to happen next and this is model formation. Very few of us read in a way where the story unfolds with us completely distant from it – in fact, maintaining distance from a story is a sign of a poor narrative. When the right story is told, or the right person is telling it, you are on the edge of your seat, hungry to know more. When it is told poorly, then you stifle a yawn and smile politely, discreetly peering at your watch as you attempt to work out the time at which you can escape.

Of course, this highlights the value of narrative for us in teaching but it also reinforces that requirement that it be more than an assemblage of rambling anecdotes, it must be a constructed narration that weaves through points in a recognisable way and giving us the ability to conjecture on its direction. O. Henry endings, the classic twist endings, make no sense unless you have constructed a mental model that can be shaken by the revelations of the last paragraphs. Harry Potter book 7 makes even less sense unless one has a model of the world in which the events of the book can be situated.

As always, this stresses the importance of educational design, where each story, each fact, each activity, is woven into the greater whole with a defined purpose and in full knowledge of how it will be used. There is nothing more distracting than someone who rambles during a lecture about things that not only seem irrelevant, but are irrelevant. Whereas a musing on something that, on first glance, appears irrelevant can lead to exploration of the narrative by students. Suddenly, they are within a Choose Your Own Adventure book and trying to work out where each step will take them.

Stories are an excellent way to link knowledge and problems. They excite, engage and educate, when used correctly. We are all hungry for stories: we are players within our own stories, observers of those of the people around us and, eventually, will form part of the greater narrative by the deeds for which we are written up in the records to come. It makes sense to use this deep and very human aspect of our intellect to try and assist with the transfer of knowledge.

ICER 2012: Day 0 (Workshops)

Posted: September 10, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , | 1 Comment

Well, it’s Sunday so it must be New Zealand (or at least it was Sunday yesterday). I attended that rarest of workshops, one where every session was interesting and made me think – a very good sign for the conference to come.

We started with an on-line workshop on Bloom’s taxonomy, classifying exam questions, with Raymond Lister from UTS. One of the best things about this for me was the discussion about the questions where we disagreed: is this application or synthesis? It really made me think about how I write my examinations and how they could be read.

We then segued into a fascinating discussion of neo-Piagetian theory, where we see the development stages that we usually associate with children in adults as they learn new areas of knowledge. In (very rough) detail, we look at whether we have enough working memory to carry out a task and, if not, weird things happen.

Students can indulge in some weird behaviours when they don’t understand what’s going on. For example, permutation programming, where they just type semi-randomly until their program compiles or works. Other examples include shotgun debugging and voodoo programming and what these amount to are the student not having a good consistent model of what works and, as a result, they are basically dabbling in a semi-magic approach.

My notes from the session contain this following excerpt:

“Bizarro” novice programmer behaviours are actually normal stages of intellectual development.
Accept this and then work with this to find ways of moving students from pre-op, to concrete op, to formal operational. Don’t forget the evaluation. Must scaffold this process!

What this translates to is that the strange things we see are just indications that students having moved to what we would normally associate with an ‘adult’ (formal operational) understanding of the area. This shoots several holes in the old “You’re born a programmer” fallacy. Those students who are more able early may just have moved through the stages more quickly.

There was also an amount of derisive description of folk pedagogy, those theories that arise during pontification in the tea room, with no basis in educational theory or formed from a truly empirical study. Yet these folk pedagogies are very hard to shake and are one of the most frustrating things to deal with if you are in educational research. One “I don’t think so” can apparently ignore the 70 years since Dewey called the classrooms prisons.

The worst thought is that, if we’re not trying to help the students to transition, then maybe the transition to concrete operation is happening despite us instead of because of us, which is a sobering thought.

I thought that Ray Lister finished the session with really good thought regarding why students struggle sometimes:

The problem is not a student’s swimming skill, it’s the strength of the torrent.

As I’ve said before, making hard things easier to understand is part of the job of the educator. Anyone will fail, regardless of their ability, if we make it hard enough for them.

Conference Blogging! (Redux)

Posted: September 8, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , | 1 Comment

I’m about to head off to another conference and I’ve taken a new approach to my blogging. Rather than my traditional “Pre-load the queue with posts” activity, which tends to feel a little stilted even when I blog other things around it, I’ll be blogging in direct response to the conference and not using my standard posting time.

I’m off to ICER, which is only my second educational research conference, and I’m very excited. It’s a small but highly regarded conference and I’m getting ready for a lot of very smart people to turn their considerably weighty gaze upon the work that I’m presenting. My paper concerns the early detection of at-risk students, based on our analysis of over 200,000 student submissions. In a nutshell, our investigations indicate that paying attention to a student’s initial behaviour gives you some idea of future performance, as you’d expect, but it is the negative (late) behaviour that is the most telling. While there are no astounding revelations in this work, if you’ve read across the area, putting it all together with a large data corpus allows us to approach some myths and gently deflate them.

Our metric is timeliness, or how reliably a student submitted their work on time. Given that late penalties apply (without exception, usually) across the assignments in our school, late submission amounts to an expensive and self-defeating behaviour. We tracked over 1,900 students across all years of the undergraduate program and looked at all of their electronic submissions (all programming code is submitted this way, as are most other assignments.) A lot of the results were not that unexpected – students display hyperbolic temporal discounting, for example – but some things were slightly less expected.

For example, while 39% of my students hand in everything on time, 30% of people who hand in their first assignment late then go on to have a blemish-free future record. However, students who hand up that first assignment late are approximately twice as likely to have problems – which moves this group into a weakly classified at-risk category. Now, I note that this is before any marking has taken place, which means that, if you’re tracking submissions, one very quick and easy way to detect people who might be having problems is to look at the first assignment submission time. This inspection takes about a second and can easily be automated, so it’s a very low burden scheme for picking up people with problems. A personalised response, with constructive feedback or a gentle question, in the zone where the student should have submitted (but didn’t), can be very effective here. You’ll note that I’m working with late submitters not non-submitters. Late submitters are trying to stay engaged but aren’t judging their time or allocating resources well. Non-submitters have decided that effort is no longer worth allocating to this. (One of the things I’m investigating is whether a reminder in the ‘late submission’ area can turn non-submitters into submitters, but this is a long way from any outcomes.)

I should note that the type of assignment work is important here. Computer programs, at least in the assignments that we set, are not just copied in from text. They are not remembering it or demonstrating understanding, they are using the information in new ways to construct solutions to problems. In Bloom’s revised taxonomic terms, this is the “Applying” phase and it requires that the student be sufficiently familiar with the work to be able to understand how to apply it.

Bloom’s Revised Taxonomy

I’m not measuring my students’ timeliness in terms of their ability to show up to a lecture and sleep or to hand up an essay of three paragraphs that barely meets my requirements because it’s been Frankenwritten from a variety of sources. The programming task requires them to look at a problem, design a solution, implement it and then demonstrate that it works. Their code won’t even compile (turn into a form that a machine can execute) unless they understand enough about the programming language and the problem, so this is a very useful indication of how well the student is keeping up with the demands of the course. By focusing on an “Applying” task, we require the student to undertake a task that is going to take time and the way in which they assess this resource and decide on its management tells us a lot about their metacognitive skills, how they are situated in the course and, ultimately, how at-risk they actually are.

Looking at assignment submission patterns is a crude measure, unashamedly, but it’s a cheap measure, as well, with a reasonable degree of accuracy. I can determine, with 100% accuracy, if a student is at-risk by waiting until the end of the course to see if they fail. I have accuracy but no utility, or agency, in this model. I can assume everyone is at risk at the start and then have the inevitable problem of people not identifying themselves as being in this area until it’s too late. By identifying a behaviour that can lead to problems, I can use this as part of my feedback to illustrate a concrete issue that the student needs to address. I now have the statistical evidence to back up why I should invest effort into this approach.

Yes, you get a lot of excuses as to why something happened, but I have derived a great deal of value from asking students questions like “Why did you submit this late?” and then, when they give me their excuse, asking them “How are you going to avoid it next time?” I am no longer surprised at the slightly puzzled look on the student’s face as they realise that this is a valid and necessary question – I’m not interested in punishing them, I want them to not make the same mistake again. How can we do that?

I’ll leave the rest of this discussion for after my talk on Monday.

A Good Friday: Student Brainstorming Didn’t Kill Me!

Posted: August 26, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | Leave a comment

We had 19 of last year’s Year 10 Tech School participants back for a brainstorming session yesterday, around the theme “What do you like about ICT/What would you say to other people about ICT.” I started them off with some warm-up exercises, as I only had three hours in total. We started with “One word to describe Tech School 2011”, “two words to describe anything you learnt or used from it”, and “three words to discuss what you think about ICT”. The last one got relaxed quickly as people started to ask whether they could extend it. We split them into tables and groups got pads of post-it notes. Get an idea, write it down, slam it on the table *thump*.

Nobody sketched Babbage and slammed it on the table. (I’m not all that surprised.)

After they had ideas all over the table, I asked them to start assembling them into themes – how would they make sentences or ideas out of this. The most excellent Justine, who did all of the hard work in setting this up (thank you!), had pre-printed some pages of images so the students could cut these out and paste them into places to convey the idea. We had four groups so we ended up with four initial posters.

Floating around, and helping me to facilitate, were Matt and Sami, both from my GC class and they helped to keep the groups moving, talking to students, drawing out questions and also answering the occasional question about the Bachelor of Computer Science (Advanced) and Grand Challenges.

We took a break for two puzzles (Eight Queens and combining the digits from 1 to 9 to equal 100 with simple arithmetic symbols) and then I split the groups up to get them to look at each other’s ideas and maybe get some new ideas to put onto another poster.

Yeah, that didn’t go quite as well. We did get some new ideas but it became obvious that we either needed to have taken a longer break, or we needed some more scaffolding to take the students forward along another path. Backtracking is a skill that takes a while to learn and, even with the graphic designer walking around trying to get some ideas going, we were tapping out a bit by the time that the finish arrived.

However, full marks to the vast majority of the participants who gave me everything that they could think of – with a good spread across schools and regions, as well as a female:male ratio of about 50%, we got a lot of great thoughts that will help us to talk to other students and let them know why they might want to go into ICT… or just come to Uni!

I didn’t let the teachers off the hook either and they gave us lots of great stuff to put into our outreach program. As a hint, I’ve never met yet a teacher at one of these events who said “Oh no, we see enough Uni people and students in the schools”, the message is almost always “Please send more students to talk to our students! Send more info!” The teachers are as, if not more, dedicated to getting students into Uni so that’s a great reminder that we’re all trying to do the same thing.

So, summary time, what worked:

  1. Putting the students into groups, armed with lots of creative materials, and asking them what they honestly thought. We got some great ideas from that.
  2. Warming them up and then getting them into story mode with associated pictures. We have four basic poster themes that we can work on.
  3. Giving everyone a small gift voucher for showing up after the fact, with no judging quality of ideas. That just appeals to my nature – I have no real idea what effect that had but I didn’t have to tell anyone that they were wrong (or less than right) because that wasn’t the aim of today.
  4. Getting teachers into a space where they could share what they needed from us as well.

What needs review or improvement:

  1. I need to look at how idea refinement and recombination might work in a tight time frame like this. I think, next time, I’ll get people to decompose the ideas to a mind map hexagon or something like that – maybe even sketch up the message graphically? Still thinking.
  2. I need more helpers. I had three and I think that a couple more would be good, as close to student age as possible.
  3. The puzzles in the middle should have naturally led to new group formation.
  4. Setting it an hour later so that everyone can get there regardless of traffic.

So, thanks again to Justine and Joh for making this work and believing in it enough to give it a try – I believe it really worked and, to be honest, far better than I thought it would but I can see how to improve it. Thanks to Matt and Sami for their help and I really hope that seeing that I actually believe all that stuff I spout in lectures wasn’t too weird!

But. of course, my thanks to the students and teachers who came along and took part in something just because we asked if they’d like to come back. Yeah, I know the motives varied but a lot of great ideas came out and I think it’ll be very helpful for everyone.

Onwards to the posters!

Musing on MOOCs

Posted: August 24, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | 1 Comment

Mark Guzdial’s blog contains a number of posts where he looks at Massive Open On-line Courses (MOOCs) but a recent one on questionable student behaviour made me think about how students act and, from the link where students sign up multiple times so that they can accumulate a ‘perfect’ score for one of their doppelgängers, why a student would go to so much trouble in a course. As the post that Mark refers to asks, is this a student retaking the course/redoing an assignment until they achieve mastery (which is highly desirable) or are they recording their attempts and finding the right answer through exhausting the search space (which is not productive and starts to look like cheating, if it isn’t actually cheating – it’s certainly against the terms of service of the courses.)

Why is this important? It’s important because MOOCs look great in terms of investment and return. Set up a MOOC and you can have 100,000 students enrol! One instructor, maybe a handful of TAs, some courseware – 100,000 students! (Some of the administrators in my building have just had to break out the smelling salts at the thought of income to expenditure ratio.) Of course, this assumes that we’re charging, which most don’t just for participation although you may get charged a fee for anything that allows you to derive accreditation. It also assumes that 100,000 students turns into some reasonable number of completions, which it also doesn’t and, as has been discussed elsewhere, plagiarism/copying is a pretty big problem.

Hang on. The course is free. It’s voluntary to sign-up to in the vast majority of cases. Why are people carrying out this kind of behaviour in a voluntary, zero-cost course? One influence is possible future accreditation, where students regard their previous efforts as a dry-run to get a high percentage outcome on a course from a prestigious institution. I’ll leave those last two words hanging there while I talk about James Joyce for a moment.

If you know of James Joyce, or you’ve read any James Joyce, you may be able to guess the question that I’m about to ask.

“Have you read and finished Ulysses?”

Joyce’s Ulysses is regarded as one of the best English-language novels of the 20th Century. However, at over 250,000 words long (that’s longer than the longest Harry Potter, by the way, and about half the count of Lord of the Rings) , full of experimental techniques, complexity and a stream-of-consciousness structure, it isn’t exactly accessible to a vast number of readers. But, because it is widely regarded as a very important novel, it is often a book that people are planning to read. Or, having started, that they plan to finish.

However, the number of people that have actually read Ulysses, all the way through and reading every word, is probably quite small. The whole ‘books I claim to have read’ effect is discussed reasonably often. From that link:

Asked if they had ever claimed to read a book when they had not, 65% of respondents said yes and 42% said they had falsely claimed to have read Orwell’s classic [1984] in order to impress. This is followed by Tolstoy’s War and Peace (31%), James Joyce’s Ulysses (25%) and the Bible (24%).

So, having possibly neither started nor finished, they claim that they have read it, because of the prestige of the work. 42% of people claim to, but haven’t read 1984, which, compared to Ulysses, is positively a pamphlet – a bus ticket aphorism in terms of relative length and readability. And we see that the other three books on the list are large, long and somewhat ponderous. (Sorry, Tolstoy, but we don’t all get locked into our dachas for 6 months when it snows.) 1984, of course, is in the public eye because of the ‘Big Brother’ associations and the on-going misinterpretation of the work as predictive, rather than as an insightful and brooding reflection of Eric’s dislike of the BBC and post-war London. (Sorry, that’s a bit glib, but I’m trying to keep it short.)

I have read Ulysses but I think it fair to say that I read it, and forced myself to complete it, for entirely the wrong reasons. Now that I enjoy the work of the Modernists far more, I’m planning to return to Ulysses and see how much I enjoy the journey this time – especially as I shall be reading it for my own reasons. But, the first time, I read it and completed it because of the prestige of the work and because I wanted to see what all the fuss was about. (Hint: it’s about a day in Dublin.)

I think there’s an intersection between the mindset that would make you claim to read a book that you had not, for reasons of prestige rather than purely for tribal membership, and that required to take a MOOC from Stanford, Harvard or Berkeley, and to falsify your progress by copying answers from other people or by solemnly duplicating your identities to accumulate enough answers to be able to ‘graduate’ summa cum laude. In this case, taking a course from one of these august institutions, especially in these days of the necessity of having a college degree for many jobs that have no professional requirement for it, is better than not. Completing assignments to a high standard, however you achieve it, may start to define your worth – this is a conjunction of prestige and tribalism that may one day allow you to become a graduate of University X (even if it is tagged as on-line or there is subsequent charging or marking load for accreditation).

And here we find our strong need for real evidence of the efficacy of the MOOC approach. Let’s assume that we solve the identity problem and can now attach work to a person reliably – how will we measure if someone is seeking mastery or is actually trying to cheat? We can ask that now – is the student who seeks questions to previous examinations testing their understanding and knowledge or conducting a brute force attack against our test bank? If MOOCs can work then the economies of scale make them a valuable tool for education but there are so many confounding factors as we try to assess these new courses: high sign-up rates with very low completion rates, high levels of plagiarism, obvious and detectable levels of gaming and all of this happening before they actually become strong alternatives to the traditional approach.

It would be easy to dismiss my comments as those of a disgruntled traditionalist but that would be wrong. What I need is evidence of what works. I have largely abandoned lectures in favour of collaborative and interactive sessions because the efficacy of the new approach became apparent – through research and evidence. Similarly for my investigation into deadlines and assessment, evidence drove me here.

If MOOCs work, then I would expect to see evidence that they do. If they don’t, then I don’t want students to sign up to something that doesn’t work, potentially at the expense of other educational opportunities that do work, any more than I want someone to stop taking their medication because someone convinces them that unverifiable alternatives are better. If MOOCs don’t quite work yet, by collecting evidence, maybe we can make them work, or part of our other courses, or produce something that benefits all of us.

It’s not about tradition or exclusivity, it’s about finding what works, which is all about collecting evidence, constructing hypotheses and testing them. Then we can find out what actually works.

More Thoughts on Partnership: Teacher/Student

Posted: August 23, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , , , | Leave a comment

I’ve just received some feedback on an abstract piece that is going into a local educational research conference. I talked about the issues with arbitrary allocation of deadlines outside of the framing of sound educational design and about how it fundamentally undermines any notion of partnership between teacher and student. The responses were very positive although I’m always wary when people staring using phrases like “should generate vigorous debate around expectations of academics” and “It may be controversial, but [probably] in a good way”. What interests me is how I got to the point of presenting something that might be considered heretical – I started by just looking at the data and, as I uncovered unexpected features, I started to ask ‘why’ and that’s how I got here.

When the data doesn’t fit your hypothesis, it’s time to look at your data collection, your analysis, your hypothesis and the body of evidence supporting your hypothesis. Fortunately, Bayes’ Theorem nicely sums it up for us: your belief in your hypothesis after you collect your evidence is proportional to how strongly your hypothesis was originally supported, modified by the chances of seeing what you did given the existing hypothesis. If your data cannot be supported under your hypothesis – something is wrong. We, of course, should never just ignore the evidence as it is in the exploration that we are truly scientists. Similarly, it is in the exploration of our learning and teaching, and thinking about and working on our relationship with our students, that I feel that we are truly teachers.

All your Bayes are belong to us. (Sorry.)

Once I accepted that I wasn’t in competition with my students and that my role was not to guard the world from them, but to prepare them for the world, my job got easier in many ways and infinitely more enjoyable. However, I am well aware that any decisions I make in terms of changing how I teach, what I teach or why I teach have to be based in sound evidence and not just any warm and fuzzy feelings about partnership. Partnership, of course, implies negotiation from both sides – if I want to turn out students who will be able to work without me, I have to teach them how and when to negotiate. When can we discuss terms and when do we just have to do things?

My concern with the phrase “everything is negotiable” is that it, to me, subsumes the notions that “everything is equivalent” and “every notion is of equal worth”, neither of which I hold to be true from a scientific or educational perspective. I believe that many things that we hold to be non-negotiable, for reasons of convenience, are actually negotiable but it’s an inaccurate slippery slope argument to assume that this means that we  must immediately then devolve to an “everything is acceptable” mode.

Once again we return to authenticity. There’s no point in someone saying “we value your feedback” if it never shows up in final documents or isn’t recorded. There’s no point in me talking about partnership if what I mean is that you are a partner to me but I am a boss to you – this asymmetry immediately reveals the lack of depth in my commitment. And, be in no doubt, a partnership is a commitment, whether it’s 1:1 or 1:360. It requires effort, maintenance, mutual respect, understanding and a commitment from both sides. For me, it makes my life easier because my students are less likely to frame me in a way that gets in the way of the teaching process and, more importantly, allows them to believe that their role is not just as passive receivers of what I deign to transmit. This, I hope, will allow them to continue their transition to self-regulation more easily and will make them less dependent on just trying to make me happy – because I want them to focus on their own learning and development, not what pleases me!

One of the best definitions of science for me is that it doesn’t just explain, it predicts. Post-hoc explanation, with no predictive power, has questionable value as there is no requirement for an evidentiary standard or framing ontology to give us logical consistency. Seeing the data that set me on this course made me realise that I could come up with many explanations but I needed a solid framework for the discussion, one that would give me enough to be able to construct the next set of analyses or experiments that would start to give me a ‘why’ and, therefore, a ‘what will happen next’ aspect.

 

Short and Sweet

Posted: August 21, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , , | 1 Comment

Well, it’s official. I’ve started to compromise my ability to work through insufficient rest. Despite reducing my additional work load, chewing through my backlog is keeping me working far too much and, as you can tell from the number and nature of the typos in these posts, it’s affecting me. I am currently reorganising tasks to see what I can continue to fit in without compromising quality, which means this week a lot of e-mail is being sent to sort out my priorities.

This weekend, I’m sitting down to brainstorm the rest of 2012 and work out what has to happen when – nothing is going to sneak up on me (again) this year.

In very good news, we have 18 students coming back for the pilot activity of “Our students, their words” where we ask students who love ICT an important question – “what do you like and why do you think someone else might like it?” We’re brainstorming with the students for all of Friday morning and passing their thoughts (as research) to a graphic designer to get some posters made. This is stage 1. Stage 2, the national campaign, is also moving – slowly but surely. This is why I really need to rest: I’m getting to the point where it’s important that I am at my best and brightest. Sleeping in and relaxing is probably the best thing I can do for the future of ICT! 🙂

Rather than be a hypocrite, I’m switching to ultra-short posts until I’m rested up enough to work properly again.

See you tomorrow!

← Older Entries
Newer Entries →