Our Obligations: Moral and Legal?

Posted: September 28, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , , | Leave a comment

Mark Guzdial raises an interesting point over at a BLOG@CACM article, namely that, if we don’t keep up to to date with contemporary practice in learning and teaching, can be considered unprofessional or even negligent or unethical? If we were surgeons who had not bothered to stay up to date then our patients, and certifying bodies, would be rightly upset. If we are teachers – then what?

The other issue Mark discusses is that of the legal requirement. The US has Title IX, which should extend the same participation rights to all genders for any education program or activity that attracts federal funding. If we do not construct activities that are inclusive (or we design activities that, by their nature, are exclusive) would we be liable under US law?

Mark’s final question is: If we know a better way to teach computing, are we professionally (and even legally) required to use it?

That is a spectacularly good question and, of course, it has no easy answer. Let me extend the idea of the surgeon by building on the doctors’ credo: primum non nocere (first, do no harm). Ultimately, it requires us to consider that all of our actions have outcomes and, in the case of medical intervention, we should be sure that we must always consider the harm that will be caused by this intervention.

Let us consider that there are two approaches that we could take in our pursuit of knowledge of learning and teaching: that of true scholarship of learning and teaching, and that of ignorance of new techniques of learning and teaching. (We’ll leave enthusiasm and ability to the side for the time being.) While this is falsely dichotomous, we can fix this by defining scholarship as starting at ‘knowing that other techniques exist and change might not kill you’, with everything else below that as ‘ignorance of new techniques’.

Now let us consider the impact of both of these bases, in terms of enthusiasm. If someone has any energy at all, then they will be able to apply techniques in the classroom. If they are more energetic then they will apply with more vigour and any effect will be amplified. If these are useful and evidentially supported techniques, then we would expect benefit. If these are folk pedagogies or traditions that have long been discredited then any vigour will be applied to an innately useless or destructive technique. In the case of an inert teacher, neither matters. It is obvious then that the minimum harm is to employ techniques that will reward vigour with sound outcomes: so we must either use validated techniques or explore new techniques that will work.

Now let us look at ability. If a teacher is ‘gifted’ (or profoundly experienced)  then he or she will be more likely to carry the class, pretty much regardless. However, what if a teacher is not so much of a star? Then, in this case, we start to become dependent once again upon the strength of the underlying technique or pedagogy. Otherwise, we risk harming our students by applying bad technique because of insufficient ability to correct it. Again, do no harm requires us to provide techniques that will survive the average or worse-than-average teacher, which requires a consideration of load, development level, reliance upon authority and so on – for student and teacher.

I believe that this argues that, yes, we are professionally bound to confirm our techniques and approaches and, if a better approach is available, evaluate it and adopt it. To do anything else risks doing harm and we cannot do this and remain professional. We are intervening with our students all the time – if we didn’t feel that our approach had worth or would change lives then we wouldn’t be doing it. If intervention and guidance are at our core then we must adopt something like the first, do no harm maxim because it gives us a clear signpost on decisions that could affect a student for life.

One of the greatest problems we face is potentially those people who are highly enthused and deeply undereducated in key areas of modern developments of teaching. As Kurt von Hammerstein-Equord would have said:

One must beware of anyone who is [undereducated] and [very enthusiastic] — [s/he] must not be entrusted with any responsibility because [s/he] will always cause only mischief.

If your best volunteer is also your worst nightmare, how do you resolve this when doing so requires you to say “This is right but you are wrong.” Can you do so without causing enormous problems that may swamp the benefit of doing so?

What about the legal issues? Do we risk heading into the murky world of compliance if we add a legal layer – will an ethical argument be enough?

What do you think about it?

The Complex Roles Of Universities In The Period Of Globalization – Altbach – Part 3

Posted: September 25, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , , | Leave a comment

To finish this triptych, I’d like to look at Altbach’s assessment of contemporary issues. Private education providers are one of the most obvious recent developments and, with the erosion of the public good motivator, this is no real surprise. It’s less of a surprise when you affix the word “Profit-making” in front of the words ‘education provider’. Given that there is growing demand for education and also given that we are blurring the lines between the institutions, it becomes easy to see why a new market has exploded for people who wish to provide education, or something like it, at a reasonable fee with a possibility of making lots and lots of money. This, however, has an impact on the public sector because it reduces the students who may have come to us for a variety of reasons, especially when the private institutions are targeting the more wealthy in some way. Suddenly, we find ourselves having to justify which kinds of knowledge we are teaching in the public sector because the type of knowledge, and the jobs it leads to, become an issue when you are competing for students inside certain professional areas. Faculties of Arts across the world are very much feeling themselves caught in this pinch. It is hard to imagine many older Universities making such a bald statement such as “There is no need for History or English Scholars”, yet by pumping resources into their professional and technical streams they are saying it through their resource distribution. If something does not provide income or attract the right market, a jaded eye is cast across it and, depending on the wealth and capacity of the institution, this leads to the shutting down of schools or entire faculties.

Why is this such a problem? Because restarting a discipline is much harder once the number of participants drops down too far. Reduce the number of people in a discipline and their shared publications and venues also shrink. Given that publication is vital to perceived success in many ways, this shrinkage will make it harder to publish OR lead to accusations of irrelevance as the overall citation level drops because there are so few people in the area. We are so heavily measured and assessed, as individuals and as universities, that we are beleaguered by league tables and beset by set publication standards. Our management structures, modes of accountability, the way that we have worked and thought for centuries are not a good fit for this new modality. This is not the golden age ramblings that I have previously pointed to as dreaming of better days – in this case, it’s true. Our systems don’t work with the new expectations.

Opening ourselves up to students from anywhere is a noble goal, and one I support wholeheartedly, but it brings great challenge. Can we pursue anything that interests us, relevant or not, and expect to meet the demands of the new century? If we can, I don’t think we can do it with the systems that we have and certainly not while we’re being measured on externally applied metrics of success. Even deciding on whether a student should be admitted or not is now a matter of school ranking, bonus points, place availability, status and, in murkier waters, the two speed entry system of public and privately-funded places in the same institution, where admitting one party may (in the worst case) prevent another from entering. As Altbach notes, our ideas of governance are changing as our scale grows and our complexity increases. Senior Professors used to set our course but now we either need or have taken on trained administrators who do not think as we do, have not had our training and, in many ways, treat us as a standard business with a strange product. We are more accountable than ever, while we wander around being randomly measured and trying to work out what it is that we need to do in order to be measured accurately and then try and perform our tasks of learning, teaching and research. How do we reconcile the community of scholars with the bureaucracies that run our institutions?

Altbach then moves on to discuss developing countries and the special challenges that they face. Many of these countries have broken links to their indigenous cultures, due to colonisation, occupation, war and civil unrest, and, when combined with the colonial trend to keep investment in higher education low, this means that many of these countries are systematically disadvantaged. Their systems are so small that expansion is hard – insufficient training grounds for new educators, delay in building and resource appropriation and the threat of instability combine to make it very hard to kickstart anything. Poverty and lack of local government resources move some of these attempts across to the ‘impossible’ category. As it becomes hard to limit enrolments, overcrowding is the norm and, while you can’t limit enrolment, you can use draconian measures to ensure that anyone who falls behind is ejected, in the hope that freeing up that slot might ease some of the crush on the resources. This is a very unforgiving approach to education: you have one chance, you blew it, goodbye. Given that this is one of the only paths out of poverty in many of these countries, and that it is very easy to fall behind in a poor and resource-starved system, this is a nasty little feedback loop. Where other institutions are built up in response to demand, these newer academies tend not to offer the same level of education and we once again have the problem of a piece of paper that is not as worthy as another: we are providing education in name only and creating yet another two-speed system. Where the job market and the educational bodies don’t keep up with each other you may have that most awful ghetto: the educated unemployed, who have invested time and money into a degree that grants them no advantage at all.

Where we are over-stretched, we tend to only do those things that generate the most benefit and this is also true in the case of these third world Universities. Teaching earns money so teaching dominates. Research is sidelined, international collaboration is sidelined and staff have no time to do anything except teach because they are trying to keep their salary coming. Unsurprisingly, this is not a stage set of excellence and advancement – these universities are falling further and further behind.

Altbach concludes by talking about the pressure that we are all under and that have made the majority of our institutions reactive, limiting our creativity to solving pressing problems in a response to external pressures. Right now, we are running so fast that we do not have time to question why we are even on this treadmill, let alone take any real steps to make serious change that is truly strategic rather than reactive. We have lost our autonomy to a degree, as well as our identity. We are enmeshed in society but in a role that favours the market forces and makes us dance in response to it. Altbach ponders what our role should be and proposes a move towards the broader public interest, moving away from market forces and towards academic autonomy.This is not the selfish “leave me alone” cry of a spoiled child, this is a recognition of the fact that we have many more things to offer than a diploma and a vocation: universities are societies of thinkers and are far more complex and diverse than our current strictures would make us appear. All universities are important, says Altbach, and it is at society’s peril that it ignores the many roles that a University can provide. Looking at us as profit-making, degree factories, or as an elite streaming system, ignores the grand public benefit of an educated society, the value of the public intellectual and the scholarly community. We deserve support, says Altbach, because serve the goals of society and the individual. Let us do our jobs properly.

I found it to be a very interesting article to read and I hope I’ve capture the essence reasonably well. I look forward to discussing it! Thanks again, RV!

The Complex Roles Of Universities In The Period Of Globalization – Altbach – Part 1

Posted: September 23, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , | Leave a comment

One of the most handy things about having a new member in a research group, especially one who is just finishing or has just finished submitting a PhD, is that they come in with an entirely new subset of the possible papers in the given discipline, which they have used to construct their theses and inform their thinking. While you will have the standard overlap of the key papers in the field, there will often be waterways that run away from the main river and it is in these diverse streams that we find new ways of thinking, even leading to these stream becoming tributaries that feed back into our main body, strengthening the overall work.

R has just sent me a reference from her thesis, a copy of Altbach’s 2011 publication in Higher Education in the World 3: New Challenges and Emerging Roles for Human and Social Development, entitled “The Complex Roles Of Universities In The Period Of Globalization”. The abstract is pithy but this quote stands out: “The academic drift of the 21st century raises concerns about the core functions of universities and how contemporary changes have affected academic missions.” This is a fascinating paper and one that I wish I turned up early because it has the same concerns as I do, and as Richard Hil did with his Whackademia book, in that we are all being asked to do more with less and it is how we do this that will decide our future, and the future of higher education. (Readers may recall that I did not agree with much of what Hil said – as I said, I wish I had read Altbach sooner because it would have made the rebuttal easier.) I’m going to cover this across a few posts because the paper has a fair bit of comment and I’d like to make some commentary!

Altbach looks at the different roles that Universities have had over time, including the different roles that they play in certain countries and how time, politics, religion, wealth and nationalism have all contributed to changing demands on the sector. There is no doubt that teaching and research make up our core functions but we can vary from country to country as to whether we are teaching technical skills, professional skills or general education at Universities. Over time, we have often been in conflict with our own societies, which can lead to great creativity but at the cost of additional load or difficult burdens. Research is equally difficult to pin down: are we talking ‘pure’ research or ‘applied’ research? Does research have to be discipline focused or can we perform research on teaching, or research on research? Does it matter where the research money comes from? Different areas inside the same university can have completely different answers to these questions so it’s little doubt that this question is still open!

Universities have been used to foster national development and identity, as Altbach mentions with German, Japanese and American examples, or as stabilising influences in the third world. We are also steadily evolving academic centres, adding courses as the ranks of the professions grow. My profession, Computer Scientist, wasn’t even a profession until the second half of the 20th century (that’s why we have so few cool awards – there is no Nobel prize in Applied Algorithmics). Immediately we see a conflict in the sense of stability and status quo required to be a national touchstone, while determining how we adapt to the changing demands of the workforce and the new professionals.

We have always been associated with knowledge as both the defenders and disseminators, ignoring secular and religious demand to not teach certain things or to state that red is black, with a focus on organisation to facilitate later retrieval. This access to knowledge also feeds in to one of our other key facets, or at least one of the most desirable, that of an intellectual centre. As academics, we have the freedom to express our ideas and, many would argue, the obligation to do so given that we have that freedom. The expertise that our staff have should be available to all in terms of interpretation and refinement of ideas and concepts but to do that we have to engage with the community. It is of little surprise that we often find ourselves involved in social and political movements, supporting other activists, providing resources and making an overall contribution to the intellectual life of our surroundings.

This is, for me, a very important point because it forces us to consider where the private individual ends and the public intellectual begins, if such a division even makes any sense. From a personal perspective, I would not raise my politics in a classroom but I would discuss issues of ethics and equality, some of which may or may not be in accord with prevailing government thought. Let me be more explicit. Yesterday, I attended a rally for Marriage Equality, as part of a reaction against the Australian Federal Government’s rejection of a bill to allow same sex marriage. I would most certainly not have advertised this event in my lectures or told my students about it because I think that there’s far too much capacity for me to influence my students to act through our relationship, which is not a discussion or political sharing but overt influence. I attended the rally as a private citizen but if my students asked me about it, because we did get photographed and videoed, then I feel that I could explain my actions within an ethical framework, which means that this is informing my role as public intellectual. My community, equality and ethical focus drives both the citizen and the academic and allows me to carry out two roles while attempting to minimise any exploitation of the power relationship that I have with my students. However, my capacity as a (notional) public intellectual requires me to have an explanation for what I did that is articulate and comprehensible. The private citizen is impassioned but the academic is both passionate and rationale, and can place the activity in a context that allows it to be shared.

But, as I always say, there is no point having a system that only works with perfect people. Altbach is talking about our institutions, which is the right focus for the paper, but the institutions are just buildings without the academics and students that fill them. I attempt to juggle my private and public self and, while sometimes I succeed more than others, I think I know what I should ‘look like’ to my institution, my peers, my students and my social groups. What will be interesting in the coming world of change for Universities is how we deal with the people who don’t work as well within the role of educator. I have no time, respect or tolerance for those of my colleagues who confuse intellectual freedom with a wanton disregard for reasonable behaviour in this privileged role. Just because we organise the knowledge doesn’t mean that we own it, nor does our mastery of intellectual pursuits (if we achieve that) make us any better than anyone else: we have merely had more opportunity but, for me, that comes with a corresponding level of responsibility. I have seen more than one academic, not at my own University I hasten to add, who has obviously been grooming a student through manipulation of the aura of competency that any decent academic can muster, where we appear wise, worldly and incredibly, staggeringly, deep on matters that are so very, very passionate and important. Altbach writes of what changes we have seen in Universities but you only have to read through the yellow press (or the FFFF00 press on the web) to see how many educators are abusing their relationship with their students and I’m not sure what this says about how the educators themselves are changing. I have heard dire tales of exploitative behaviour in the 70s and 80s in my country – “A for a lay” unpleasantness and similar. When we talk of our intellectual freedoms, our influence on the world as national stabilisers and centres of knowledge, it is important to remember that the components of these institutions are merely people. As we increase the stresses on the organisations, so too do we distribute this across people and, given that people are already failing some key moral and disciplinary requirements, any discussion of what our role should become will have to take into account the fact that we are building a system from people, to work with other people.

This Is Your Captain Speaking: Turn Off Your Gadgets

Posted: September 22, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , | 1 Comment

“Ladies and gentlemen, boys and girls,

This is your Captain speaking. Shortly,I’m going to push on a set of levers that will allow fuel to pour into the steel cylinders affixed to the side of the plane and, in the presence of well-engineered flame, create a series of small and controlled explosions that will allow us to lift roughly 400 tonnes of metal and flesh into the air and propel you towards your destination.

As part of this, when the plane heads down the runway, we will pass a threshold known as V1. Do you know what V1 means? V1 is the takeoff decision speed  and is the speed at which we will try and take off, even if one engine fails. Basically, V1 is the speed at which we are travelling so fast that we are safest in the air – we are beyond the realm of air brakes or the rather amusingly useless brakes on the wheels. You know that mechanical brakes can evaporate on trucks when they’re going too fast, right? Think an even faster “pfft, bloooo” for planes.

Having gone through V1, we will reach V2. That is the speed at which, having committed to take-off, we will attempt to rotate the nose up and we will safely be able to take off, even if one engine is down. You know this speed. We trundle down the runway and you might miss V1 but the moment we rotate the nose up, that’s V2.

If I can’t easily find out when either of these speeds are, we’re stuffed. I need the read-outs in front of me to give me a reliable idea of these speeds or my co-pilots and I will be working on guesswork and you, seriously, do not want that.

There are two points when you are seriously vulnerable in an aircraft: takeoff and landing. During both of these moments, our proximity to the ground and reduced speed combine to form a major liability: any misjudgement at this point can lead to catastrophe because we have not got any time to recover from disaster. This, of course, is why we ask you to turn off every single possible source of interference to out aircraft systems, which includes phones, iPods, iPads, MP3 players, Kindles, Kobo readers, whatever. We want to have the best chance possible to let the plane tell us everything that it can.

Some of you don’t turn the gadgets off and, as your Captain, let me berate both you and the people who educated you. You, because you ignored the legal requirement to comply with my instructions and your teachers, because they failed to adequately instruct you in the importance of cause and effect, personal responsibility, and anecdotal evidence.

If you, as a person, opt to leave your gear on despite being asked not to, you are saying to the roughly 500 other passengers that your need to read a book in an electronic form, or watch a movie, trumps their fundamental right to personal safety. Now, we’re not 100% sure that this will cause a problem but, as we’re also not sure that it won’t interfere with our systems and we do need to know lots of stuff about the aeroplane at these critical times, we ask you to switch this gear off. Will it cause an accident? Probably not. Is it safe? We don’t know. Right now, you’re not demonstrating an adequate knowledge of cause and effect or personal responsibility.

Oh, so your Uncle Willie left his mobile on and nothing happened? Great! Fantastic! Was it this type of plane? Same avionics? Was there unforeseen confusion in the flight deck that no-one mentioned (Probably not but you don’t know.) Hey,  I hear Uncle Willie drove through an intersection once against the red lights at 100 mph – why don’t you try that?  Anecdotal evidence, especially one exceptional case, proves nothing.

Actions have repercussions but this doesn’t mean that there will always be a 1:1 match-up between them. If mobile phones always crashed planes, we’d search you and confiscate them. It’s the possible and unlikely interaction of plane and gadget that we’re worried about and this is why we sincerely hope that your teachers have managed to get this idea through to you, along with the fact that rationalisation doesn’t equal reason and one contrary exemplar does not state a uniform case.

Let me remind you that we will shortly be flying in a 400 tonne piece of metal that hurtles through the sky at 650 mph on top of 2-4 engines of burning flame.

Do you actually want to make it harder for me to control this?

Think about the possible impact of your actions, comply with crew directions and, for a few minutes at the start and end of the flight, do what I ask you to do and turn off your gadgets. It might not do anything, but it might give you a chance to be irritated by a similar announcement on a subsequent flight.

Thank you for your attention.”

ICER 2012 Day 3 Research Paper Session 5

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , | Leave a comment

The last of the research paper sessions and, dear reader, I am sure that you are as glad as I that we are here. Reading about an interesting conference that you didn’t attend is a bit like receiving a message from a friend talking about how he kissed the person that you always loved from afar. Thanks for the information but I would rather have been there myself.

This session opened with “Toward a Validated Computing Attitudes Survey” (Allison Elliott Tew, Brian Dorn and Oliver Schneider), where the problems with negative perceptions of the field and hostile classroom environments, combined with people thinking that they would be no good at CS, conspire to prevent students coming in to, or selecting, our discipline. The Computing Attitudes Survey was built, with major modification, from the Colorado Learning Attitudes about Science Survey (CLASS, pronounced C-LASS). To adapt the original survey, some material was just copied across with a word change (computer science replacing physics), some terminology was changed (algorithm for formula) and some discipline specific statements were added. Having established an expert opinion basis for the discipline specific content, students can now see how much they agree with the experts.

There is, as always, the rip of contentious issues. “You have to know maths to be able to program” was a three-way split within the expert group as to who agreed, disagreed or was neutral. What was interesting, and what I’ll be looking at in future, is the evidence of self-defeating thought in many answers (no, not questions. The questions weren’t self-defeatist but the answers often were.) What was also interesting is that attitudes seem to get worse in the CLASS instrument after you take the course!

Confidence, as simple as “I think I can do this”, plays a fundamental part in determining how students will act. Given the incredibly difficult decisions that a student faces when selecting their degree or concentration, it is no surprise that anyone who thinks “Computing is too hard for me” or “Computing is no use to me” will choose to do something else.

The authors are looking for volunteers where they can run these trials again so, after you’ve read their paper, if you’re interested, you should probably e-mail them.

“A Statewide Survey on Computing Education Pathways and Influences: Factors in Broadening Participation in Computing” (Mark Guzdial, Barbara Ericson, Tom McKlin and Shelly Engelman)

The final research paper in the conference dealt with the final evaluation of the Georgia Computes! initiative, which had run from October 2006 to August of this year. This multi-year project cannot be contained in my nervous babbling but I can talk about the instrument that was presented. Having run summer camps, weekend workshops, competitions, teacher workshops, a teachers’ lending library, first year engagement and seeded first-year summer camps (whew!), the question was: What had been the impact of Georgia Computes! ? What factors influence undergrad enrolment into intro CS courses?

There were many questions and results presented but I’d like to focus on the top four reasons given, from survey, as to why students weren’t going to undertake a CS Major or Minor:

  1. I don’t want to do the type of work
  2. Little interest in the subject matter
  3. Don’t enjoy Computing Courses
  4. Don’t have confidence that I would succeed.

Looking at those points, after a state-wide and highly successful campaign over 6 years has finished, it is very, very sobering for me. What these students are saying is that they cannot see the field as attractive, interesting, enjoyable or that they are capable. But these are all aspects that we can work on, although some of these will require a lot of work.

Two further things that Barb said really struck me. Firstly, that if you take into account encouragement and ability, that men will tend to be satisfied and continue on if they receive either or both – the factors are not separable for men – but that women and minorities need encouragement in order to feel satisfied and to convince them to keep going. Secondly, when it comes to giving encouragement, male professors are just as effective as female professors in terms of giving encouragement to women.

As a male lecturer, who is very, very clearly aware of the demographic disgrace that is the under-representation of women in CS, this first fact gives me a partial strategy to increase retention (and reinforces a believe I have held anecdotally for some time) but the second fact gives me the agency to assist in this process, as well as greater hope for a steadily increasing female cohort over time.

Overall, a very positive note on which to finish the session papers!

ICER 2012 Day Research Paper Session 4

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , | Leave a comment

This session kicked off with “Ability to ‘Explain in Plain English’ Linked to Proficiency in Computer-based Programming”,  (Laurie Murphy, Sue Fitzgerald, Raymond Lister and Renee McCauley (presenting)). I had seen a presentation along these lines at SIGCSE and this is an excellent example of international collaboration, if you look at the authors list. Does the ability to explain code in plain English correlate with ability to solve programming problems? The correlation appears to be there, whether or not we train students in Explaining in Plain English or not, but is this causation?

This raises a core question, addressed in the talk: Do we need to learn to read (trace) code before we learn to write code or vice versa? The early reaction of the Leeds group was that reading code didn’t amount to testing whether students could actually write code. Is there some unknown factor that must be achieved before either or both of these? This is a vexing question as it raises the spectre of whether we need to factor in some measure of general intelligence, which has not been used as a moderating factor.

Worse, we now return to that dreadful hypothesis of “programming as an innate characteristics”, where you were either born to program or not. Ray (unsurprisingly) believes that all of the skills in this area (EIPE/programming) are not innate and can be taught. This then raises the question of what the most pedagogically efficient way is to do this!

How Do Students Solve Parsons Programming Problems? — An Analysis of Interaction Traces
Juha Helminen (presenting), Petri Ihantola, Ville Karavirta and Lauri Malmi

This presentation was of particular interest to me because I am currently tearing apart my 1,900 student data corpus to try and determine the point at which students will give up on an activity, in terms of mark benefit, time expended and some other factors. This talk, which looked at how students solved problems, also recorded the steps and efforts that they took in order to try and solve them, which gave me some very interesting insights.

A Parsons problem is one where, given code fragments a student selects, arranges and composes a program in response to a question. Not all code fragments present will be required in the final solution. Adding to the difficulty, the fragments require different indentation to assert their execution order as part of block structure. For those whose eyes just glazed over, this means that it’s more than selecting a line to go somewhere, you have to associate it explicitly with other lines as a group. Juha presented a graph-based representation of the students’ traversals of the possible solutions for their Parsons problem. Students could ask for feedback immediately to find out how their programs were working and, unsurprisingly, some opted for a lot of “Am I there yet” querying. Some students queried feedback as much as 62 times for only 7 features, indicative of permutation programming, with very short inter-query intervals. (Are we there yet? No. Are we there yet? No. Are we there yet? No. Are we there yet? No.)

The primary code pattern of development was linear, with block structures forming the first development stages, but there were a lot of variations. Cycles (returning to the same point) also occurred in the development cycle but it was hard to tell if this was a deliberate reset pattern or a one where permutation programming had accidentally returned the programmer to the same state. (Asking the students “WHY” this had occurred would be an interesting survey question.)

There were some good comments from the audience, including the suggestion of correlating good and bad states with good and bad outcomes, using Markov chain analysis to look for patterns. Another improvement suggested was recording the time taken for the first move, to record the impact (possible impact) of cognition on the process. Were students starting from a ‘trial and error’ approach or only after things went wrong?

Tracking Program State: A Key Challenge in Learning to Program
Colleen Lewis (presenting, although you probably could have guessed that)

This paper won the Chairs’ Award for the best paper at the Conference and it was easy to see why. Colleen presented a beautifully explored case study of an 11 year old boy working on a problem in the Scratch programming language and trying to work out why he couldn’t draw a wall of bricks. By capturing Kevin’s actions, in code, his thoughts, from his spoken comments, we are exposed to the thought processes of a high achieving young man who cannot fathom why something isn’t working.

I cannot do justice to this talk by writing about something that was primarily visual, but Colleen’s hypothesis was that Kevin’s attention to the state (variables and environmental settings over which the program acts) within the problem is the determining factor in the debugging process. Once Kevin’s attention was focused on the correct problem, he solved it very quickly because the problem was easy to solve. Locating the correct problem required him to work through and determine which part of the state was at fault.

Kevin has a pile of ideas in his head but, as put by duSessa and Sherin (98), learning is about reliably using the right ideas in the correct context. Which of Kevin’s ideas are being used correctly at any one time? The discussion that followed talked about a lot of the problems that students have with computers, in that many students do not see computers as actually being deterministic. Many students, on encountering a problem, will try exactly the same thing again to see if the error occurs again – this requires a mental model that we expect a different set of outcomes with the same inputs and process, which is a loose definition of either insanity or nondeterminism. (Possibly both.)

I greatly enjoyed this session but the final exemplar, taking apart a short but incredibly semantically rich sequence and presenting it with a very good eye for detail, made it unsurprising that this paper won the award. Congratulations again, Colleen!

ICER 2012 Day 2 Discussion Papers Session 2

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , | Leave a comment

This is a brief note on this session as these papers are presented to the community with the intention of sparking discussion and, in this case, one of the most interesting issues that arose was the use for reference of the first of a pair of papers, where the first paper asserted a finding and the second then retracted it. This is not to say that the actual papers presented themselves weren’t interesting (far from it) but you can read about them in the proceedings and this particular session raised yet another reason to come to ICER: because this is where a lot of the authors are.

In this case, one of the authors on the retraction paper very politely identified himself and then pointed out why the paper in question that the presenting authors were referring to had then been followed-up with a paper that illustrated some of the problems in the original work. (I am trying quite hard to avoid potentially embarrassing anyone so please excuse how circumspect I am being.)

The reception of the actual discussion paper was, unsurprisingly, framed in the revelation that a supporting paper had been undermined and the questions revolved around issues with metrics and how the authors had addressed the (so-called) possible Hawthorne effect issues.

But this is exactly what these kind of paper sessions are for. This is a place to present ideas for the community and now the authors can go back, rework their approach on stronger soil and come back with something stronger. Yes, there is no doubt that they would much rather have not built upon that paper but imagine how much worse it would have been had this made it (undetected) to the journal stage!

 

ICER 2012 Day 2 Research Session 3

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , | Leave a comment

The session kicked off with “The Abstraction Transition Taxonomy: Developing Desired Learning Outcomes through the Lens of Situated Cognition”, (Quintin Cutts (presenting), Sarah Esper, Marlena Fecho, Stephen Foster and Beth Simon) and the initial question: “Do our learning outcomes for programming classes match what we actually do as computational thinkers and programmers?” To answer this question, we looked Eric Mazur’s Peer Instruction, an analysis of PU questions as applied to a CS principles pilot course, and then applied the Abstraction Transition Taxonomy (ATT) to published exams, with a wrap of observations and ‘where to from here’.

Physicists have, some time ago, noticed that their students can plug numbers into equations (turn the handle, so to speak) but couldn’t necessarily demonstrate that they understood things: they couldn’t demonstrate that that they thought as physicists should. (The Force Concept Inventory was mentioned here and, if you’re not familiar, it’s a very interesting thing to look up.) To try and get students who thought as physicists, Mazur developed Peer Instruction (PI), which had pre-class prep work, in-class questions, followed by voting, discussion and re-voting, with an instructor leading class-wide discussion. These activities prime the students to engage with the correct explanations – that is, the way that physicists think about and explain problems.

Looking at Computer Science, many CS people use the delivery of a working program as a measure of the correct understanding and appropriate use of programming techniques.

Given that generating a program is no guarantee of understanding, which is sad but true given the existence of the internet, other students and books. We could try and force a situation where students are isolated from these support factors but this then leads us back to permutation programming, voodoo code and shotgun debugging unless the students actually understand the task and how to solve it using our tools. In other words, unless they think as Computer Scientists.

UCSD had a CS Principles Pilot course that used programming to foster computational thinking that was aimed at acculturation into the CS ‘way’ rather than trying to create programmers. The full PI implementation asked students to reason about their programs, through exploratory homework and a PI classroom, with some limited time traditional labs as well. While this showed a very positive response, the fear was that this may have been an effect of the lecturers themselves so analysis was required!

By analysing the PI questions, a taxonomy was developed that identified abstraction levels and the programming concepts within them. The abstraction levels were “English”, “Computer Science Speak” and “Code”. The taxonomy was extended with the transitions between these levels (turning an English question into code for example is a 1-3 transition, if English is abstraction level 1 and Code 3. Similarly, explain this code in English is 3-1). Finally, they considered mechanism (how does something work) and rationale (why did we do it this way)?

Analysing the assignment and assessment questions to determine what was being asked, in terms of abstraction level and transitions, and whether it was mechanism or rationale, revealed that 21% of the in-class multiple choice questions were ‘Why?’ questions but there actually very few ‘Why?’ questions in the exam. Unsurprisingly, almost every question asked in the PI framework is a ‘Why?’ question,  so there should be room for improvement in the corresponding examinations. PI emphasises the culture of the discipline through the ‘Why?’ framing because it requires acculturation and contextualisation to get yourself into the mental space where a Rationale becomes logical.

The next paper “Subgoal-Labeled Instructional Material Improves Performance and Transfer in Learning to Develop Mobile Applications”, Lauren Margulieux, Mark Guzdial and Richard Catrambone, dealt with mental models and how the cognitive representation of an action will affect both the problem state and how well we make predictions. Students have so much to think about – how do they choose?

The problem with just waiting for a student to figure it out is high cognitive load, which I’ve referred to before as helmet fire. If students become overwhelmed they learn nothing, so we can explicitly tell students and/or provide worked examples. If we clearly label the subgoals in a worked example, students remember the subgoals and the transition from one to another. The example given here was an Android App Inventor worked example, one example of which had no labels, the other of which had subgoal labels added as overlay callouts to the movie as the only alteration. The subgoal points were identified by task analysis – so this was a very precise attempt to get students to identify the important steps required to understand and complete the task.

(As an aside, I found this discussion very useful. It’s a bit like telling a student that they need comments and so every line has things like “x=3; //x is set to 3” whereas this structured and deliberate approach to subgoal definition shows students the key steps.)

In the first experiment that was run, the students with the subgoals (and recall that this was the ONLY difference in the material) had attempted more, achieved more and done it in less time. A week later, they still got things right more often. In the second experiment, a talk-aloud experiment, the students with the subgoals discussed the subgoals more, tried random solution strategies less and wasted less effort than the other group. This is an interesting point. App Inventor allows you to manipulate blocks of code and the subgoal group were less likely to drag out a useless block to solve the problem. The question, of course, is why. Was it the video? Was it the written aspects? Was it both?

Students appear to be remembering and using the subgoals and, as was presented, if performance is improving, perhaps the exact detail of why it’s happening is something that we wish to pursue but, in the short term, we can still use the approach. However, we do have to be careful with how many labels we use as overloading visual cues can lead to confusion, thwarting any benefit.

The final paper in the session was “Using collaboration to overcome disparities in Java experience”, Colleen Lewis (presenting), Nathaniel Titterton and Michael Clancy. This presented the transformation of a a standard 3 Lecture, 2 hours of lab and 1 discussion hour course into a 1 x 1 hour lecture with 2 x 3 hour labs, with the labs now holding the core of the pedagogy. Students are provided feedback through targeted tutoring, using on-line multiple choices for the students to give feedback and assist the TAs. Pair programming gives you someone to talk to before you talk to the TA but the TA can monitor the MCQ space and see if everyone is having a problem with a particular problem.

This was addressing a problem in a dual speed entry course, where some students had AP CS and some didn’t, therefore the second year course was either a review for those students who had Java (from AP CS) or was brand new. Collaboration and targeted support was aimed at reducing the differences between the cohorts and eliminate disadvantage.

Now, the paper has a lot of detail on the different cohorts, by intake, by gender, by retention pattern, but the upshot is that the introduction of the new program reduced the differences between those students who did and did not have previous Java experience. In other words, whether you started at UCB in CS 1 (with no AP CS) or CS  1.5 (with AP CS), the gap between your cohorts shrank – which is an excellent result. Once this high level of collaboration was introduced, the only factor that retained any significant difference was the first exam, but this effect disappeared throughout the course as students received more exposure to collaboration.

I strongly recommend reading all three of these papers!

The Narrative Hunger: Stories That Meet a Need

Posted: September 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , | Leave a comment

I have been involved in on-line communities for over 20 years now and, apparently, people are rarely surprised when they meet me. “Oh, you talk just like you type.” is the effective statement and I’m quite happy with this. While some people adopt completely different personae on-line, for a range of reasons, I seem to be the same. It then comes as little surprise that I am as much of storyteller in person as I am online. I love facts, revel in truth, but I greatly enjoying putting them together into a narrative that conveys the information in a way that is neither dry nor dull. (This is not to say that the absence of a story guarantees that things must be dry and dull but, without a focus on those elements of narrative that appeal to common human experience, we always risk this outcome.)

One of Katrina’s recent posts referred to the use of story telling in education. As she says, this can be contentious because:

stories can be used to entertain students, to have them enjoy your lectures, but are not necessarily educational.

The shibboleth of questionable educational research is often a vaguely assembled study, supported by the conjecture that the “students loved it”, and it is very easy to see how story telling could fall into this. However, we as humans are fascinated by stories. We understand the common forms even where we have not read Greek drama or “The Hero With a Thousand Faces”. We know when stories ring true and when they fall flat. Searching the mental engines of our species for the sweet spots that resonate across all of us is one way to convey knowledge in a more effective and memorable way. Starting from this focus, we must then observe our due diligence in making sure that our story framework contains a worthy payload.

Not all stories are of the same value.

I love story telling and I try to weave together a narrative in most of my lectures, even down to leaving in sections where deliberate and tangential diversion becomes part of the teaching, to allow me to contrast a point or illuminate it further by stripping it of its formal context and placing it elsewhere. After all, an elephant next to elephants is hardly memorable but an elephant in a green suit, as King of a country, tends to stick in the mind.

The power of the narrative is that it involves the reader or listener in the story. A well-constructed narrative leads the reader to wonder about what is going to happen next and this is model formation. Very few of us read in a way where the story unfolds with us completely distant from it – in fact, maintaining distance from a story is a sign of a poor narrative. When the right story is told, or the right person is telling it, you are on the edge of your seat, hungry to know more. When it is told poorly, then you stifle a yawn and smile politely, discreetly peering at your watch as you attempt to work out the time at which you can escape.

Of course, this highlights the value of narrative for us in teaching but it also reinforces that requirement that it be more than an assemblage of rambling anecdotes, it must be a constructed narration that weaves through points in a recognisable way and giving us the ability to conjecture on its direction. O. Henry endings, the classic twist endings, make no sense unless you have constructed a mental model that can be shaken by the revelations of the last paragraphs. Harry Potter book 7 makes even less sense unless one has a model of the world in which the events of the book can be situated.

As always, this stresses the importance of educational design, where each story, each fact, each activity, is woven into the greater whole with a defined purpose and in full knowledge of how it will be used. There is nothing more distracting than someone who rambles during a lecture about things that not only seem irrelevant, but are irrelevant. Whereas a musing on something that, on first glance, appears irrelevant can lead to exploration of the narrative by students. Suddenly, they are within a Choose Your Own Adventure book and trying to work out where each step will take them.

Stories are an excellent way to link knowledge and problems. They excite, engage and educate, when used correctly. We are all hungry for stories: we are players within our own stories, observers of those of the people around us and, eventually, will form part of the greater narrative by the deeds for which we are written up in the records to come. It makes sense to use this deep and very human aspect of our intellect to try and assist with the transfer of knowledge.

ICER 2012 Research Paper Session 2

Posted: September 13, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | 3 Comments

Ok, true confession time. My (and Katrina’s) paper was in this session and I’ll write this up separately. So this session consisted of “Adapting Disciplinary Commons Model: Lessons and Results from Georgia” (Brianna Morrison, Lijun Ni and Mark Guzdial) and… another paper. 🙂

The goals of the original disciplinary commons were:
  • To document and share knowledge about student learning in CS classrooms
  • To establish practices for the scholarship of teaching by making it public, peer-reviewed and amenable for public use. (portfolio model)
While the first goal was achieved, the second wasn’t as, although portfolios were produced, people just wanted to keep them private. However, they did
develop a strong and vibrant community, with associated change of practice as a result of participation. The next stage was a Disciplinary Commons for Computing Educators (DCCE, for Georgia), with the adaptation that this apply to both High School teachers AND university-level educators.
The new goals were:
  1. Creating community
  2. Sharing resources and knowledge of how things are taught in other contexts.
  3. Supporting student recruitment within the high school environment.
I was interested to learn that there is no Computer Science teaching certificate in Georgia, hence a teacher must be certified in another discipline, such as Mathematics, Science, and Business being most likely. (I believe this is what was said although on reviewing my notes, I find this a little confusing. I’m assuming that this is due to the transition into the Georgia teaching framework.)
The community results were very interesting, as the initial community formed where one person was the hub – a network but not a robust one! After working on this in year 3, a much more evenly distributed group was formed that could survive a few people dropping out.  Given that many of the students in the program had no (or very few) peers in the home university, these networks were crucial to giving them important information. Teachers who work in isolation need supporting networks – you can see what else someone does, and ask how they do it.
I love these community-building projects and the network example gave one of the fantastic insights into why regular progress and impact checks can make the difference between an ok project and a highly successful one. Identifying that a network based on one (hub) person is unstable and altering your practices to make the network graph more heavily meshed is an excellent adaptation that reinforces the key focus on this project: creating community.
I read a design magazine called Desktop, much to the amusement of my more design-oriented friends, and one of the smaller regular features is that of the desks and working environments of professional designers. As I try to learn more about this area, this helps give me some insight into the community and, because of this, it accelerates my development. The community project described in this paper allows people who are already trying to hold a presence in a tricky and evolving area by connecting them with people who have similar issues and joining all of them to a shared repository of knowledge and experience. It would be great to see more programs like this.
← Older Entries
Newer Entries →