ITiCSE 2014, Day 3, Session 6A, “Digital Fluency”, #ITiCSE2014 #ITiCSE

I'm at the Ångstrom Laboratory of Uppsala so this portrait hangs in the main hall. Hi, Anders!

I’m at the Ångstrom Laboratory of Uppsala so this portrait hangs in the main hall. Hi, Anders!

The first paper was “A Methodological Approach to Key Competences in Informatics”, presented by Christina Dörge. The motivation for this study is moving educational standards from input-oriented approaches to output-oriented approaches – how students will use what you teach them in later life. Key competencies are important but what are they? What are the definitions, terms and real meaning of the words “key competencies”? A certificate of a certain grade or qualification doesn’t actually reflect true competency is many regards. (Bologna focuses on competencies but what do really mean?) Competencies also vary across different disciplines as skills are used differently in different areas – can we develop a non-normative approach to this?

The author discussed Qualitative Content Analysis (QCA) to look at different educational methods in the German educational system: hardware-oriented approaches, algorithm-oriented, application-oriented, user-oriented, information-oriented and, finally, system-oriented. The paradigm of teaching has shifted a lot over time (including the idea-oriented approach which is subsumed in system-oriented approaches). Looking across the development of the paradigms and trying to work out which categories developed requires a coding system over a review of textbooks in the field. If new competencies were added, then they were included in the category system and the coding started again. The resulting material could be referred to as “Possible candidates of Competencies in Informatics”, but those that are found in all of the previous approaches should be included as Competencies in Informatics. What about the key ones? Which of these are found in every part of informatics: theoretical, technical, practical and applied (under the German partitioning)? A key competency should be fundamental and ubiquitous.

The most important key competencies, by ranking, was algorithmic thinking, followed by design thinking, then analytic thinking (must look up the subtle difference here). (The paper contains all of the details) How can we gain competencies, especially these key ones, outside of a normative model that we have to apply to all contexts? We would like to be able to build on competencies, regardless of entry point, but taking into account prior learning so that we can build to a professional end point, regardless of starting point. What do we want to teach in the universities and to what degree?

The author finished on this point and it’s a good question: if we view our progression in terms of competency then how we can use these as building blocks to higher-level competencies? THis will help us in designing pre-requsitites and entry and exit points for all of our educational design.

The next talk was “Weaving Computing into all Middle School Disciplines”, presented by Susan Rodger from Duke. There were a lot of co-authors who were undergraduates (always good to see). The motivation for this project was there are problems with CS in the K-12 grades. It’s not taught in many schools and definitely missing in many high schools – not all Unis teach CS (?!?). Students don’t actually know what it is (the classic CS identify problem). There are also under-represented groups (women and minorities). Why should we teach it? 21st century skills, rewordings and many useful skills – from

Schools are already content-heavy so how do we convince people to add new courses? We can’t really so how about trying to weave it in to the existing project framework. Instead of doing a poster or a PowerPoint  prevention, why not provide an animations that’s interactive in some way and  that will involve computing. One way to achieve this is to use Alice, creating interactive stories or games, learning programming and computation concepts in a drag-and-drop code approach. Why Alice? There are many other good tools (Greenfoot, Lego, Scratch, etc) – well, it’s drag-and-drop, story-based and works well for women. The introductory Alice course in 2005 started to attract more women and now the class is more than 50% women. However, many people couldn’t come in because they didn’t have the prerequisites so the initiative moved out to 4th-6th grade to develop these skills earlier. Alice Virtual Worlds excited kids about computing, even at the younger ages.

The course “Adventures in Alice Programming” is aimed at grades 5-12 as Outreach, without having to use computing teachers (which would be a major restriction). There are 2-week teacher workshops where, initially, the teachers are taught Alice for a week, then the following week they develop lesson plans. There’s a one-week follow-up workshop the following summer. This initiative is funded until Summer, 2015, and has been run since 2008. There are sites: Durham, Charleston and Southern California.  The teachers coming in are from a variety of disciplines.

How is this used on middle and high schools by teachers? Demonstrations, examples, interactive quizzes and make worlds for students to view. The students may be able to undertake projects, take and build quizzes, view and answer questions about a world, and the older the student, the more they can do.

Recruitment of teachers has been interesting. Starting from mailing lists and asking the teachers who come, the advertising has spread out across other conferences. It really helps to give them education credits and hours – but if we’re going to pay people to do this, how much do we need to pay? In the first workshop, paying $500 got a lot of teachers (some of whom were interested in Alice). The next workshop, they got gas money ($50/week) and this reduced the number down to the more interested teachers.

There are a lot of curriculum materials available for free (over 90 tutorials) with getting-started material as a one-hour tutorial showing basic set-up, placing objects, camera views and so on. There are also longer tutorials over several different stories. (Editor’s note: could we get away from the Princess/Dragon motif? The Princess says “Help!” and waits there to be rescued and then says “My Sweet Prince. I am saved.” Can we please arm the Princess or save the Knight?) There are also tutorial topics on inheritance, lists and parameter usage. The presenter demonstrated a lot of different things you can do with Alice, including book reports and tying Alice animations into the real world – such as boat trips which didn’t occur.

It was weird looking at the examples, and I’m not sure if it was just because of the gender of the authors, but the kitchen example in cooking with Spanish language instruction used female characters, the Princess/Dragon had a woman in a very passive role and the adventure game example had a male character standing in the boat. It was a small sample of the materials so I’m assuming that this was just a coincidence for the time being or it reflects the gender of the creator. Hmm. Another example and this time the Punnett Squares example has a grey-haired male scientist standing there. Oh dear.

Moving on, lots of helper objects are available for you to use if you’re a teacher to save on your development time which is really handy if you want to get things going quickly.

Finally, on discussing the impact, one 200 teachers have attend the workshops since 2008, who have then go on to teach 2900 students (over 2012-2013). From Google Analytics, over 20,000 users have accessed the materials. Also, a number of small outreach activities, Alice for an hour, have been run across a range of schools.

The final talk in this session was “Early validation of Computational Thinking Pattern Analysis”, presented by Hilarie Nickerson, from University of Colorado at Boulder. Computational thinking is important and, in the US, there have been both scope and pedagogy discussions, as well as instructional standards. We don’t have as much teacher education as we’d like. Assuming that we want the students to understand it, how can we help the teachers? Scalable Game Design integrates game and simulation design into public school curricula. The intention is to broaden participation for all kinds of schools as after-scjool classes had identified a lot of differences in the groups.

What’s the expectation of computational thinking? Administrators and industry want us to be able to take game knowledge and potentially use it for scientific simulation. A good game of a piece of ocean is also a predator-prey model, after all. Does it work? Well, it’s spread across a wide range of areas and communities, with more than 10,000 students (and a lot of different frogger games). Do they like it? There’s a perception that programming is cognitively hard and boring (on the congnitive/affective graph ranging from easy-hard/exciting-boring) We want it to be easy and exciting. We can make it easier with syntactic support and semantic support but making it exciting requires the students to feel ownership and to be able to express their creativity. And now they’re looking at the zone of proximal flow, which I’ve written about here. It’s good see this working in a project first, principles first model for these authors. (Here’s that picture again)

Figure from A. Repenning, "Programming Goes to School", CACM, 55, 5, May, 2012.

Figure from A. Repenning, “Programming Goes to School”, CACM, 55, 5, May, 2012.

The results? The study spanned 10,000 students, 45% girls and 55% boys (pretty good numbers!), 48% underrepresented, with some middle schools exposing 350 students per year. The motivation starts by making things achievable but challenging – starting from 2D basics and moving up to more sophisticated 3D games. For those who wish to continue: 74% boys, 64% girls and 69% of minority students want to continue. There are other aspects that can raise motivation.

What about the issue of Computing Computational Thinking? The authors have created a Computational Thinking Pattern Analysis (CTPA) instrument that can track student learning trajectories and outcomes. Guided discovery, as a pedagogy, is very effective in raising motivation for both genders, where direct instruction is far less effective for girls (and is also less effective for boys).

How do we validate this? There are several computational thinking patterns grouped using latent semantic analysis. One of the simpler patterns for a game is the pair generation and absorption where we add things to the game world (trucks in Frogger or fish in predator/prey) and then remove them (truck gets off the screen/fish gets eaten). We also need collision detection. Measuring skill development across these skills will allow you to measure it in comparison to the tutorial and to other students. What does CTPA actually measure? The presence of code patterns that corresponded to computational thinking constructs suggest student skill with computational thinking (but doesn’t prove it) and is different from measuring learning. The graphs produced from this can be represented as a single number, which is used for validation. (See paper for the calculation!)

This has been running for two years now, with 39 student grades for 136 games, with the two human graders shown to have good inter-rater consistency. Frogger was not very heavily correlated (Spearman rank) but Sokoban, Centipede and the Sims weren’t bad, and removing design aspects of rubrics may improve this.

Was their predictive validity in the project? Did the CTPA correlate with the skill score of the final game produced? Yes, it appears to be significant although this is early work. CTPA does appear to be cabal of measuring CT patterns in code that correlate with human skill development. Future work on this includes the refinement of CTPA by dealing with the issue of non-orthogonal constructs (collisions that include generative and absorptive aspects), using more information about the rules and examining alternative calculations. The group are also working not oils for teachers, including REACT (real-time visualisations for progress assessment) and recommend possible skill trajectories based on their skill progression.

Let the Denial Begin

It is an awful fact that women are very underrepresented in my discipline, Computer Science, and as an aggregate across my faculty, which includes Engineering and Mathematics (so we’re the Technology, Engineering and Mathematics of STEM). I have heard almost every tired and discredited excuse for why this is the case but what has always angered me is the sheer weight of resistance to any research that (a) clearly demonstrates that bias exists to explain why this occurs, (b) identifies how performance can be manipulated through preconceptions and (c) requires people to consider that we are all more similar than current representation would indicate.

Yes, if I were to look around and say “Women are not going to graduate in large numbers because I see so few of them” then I would be accurate and yet, at the same time, completely missing the point. If I were to turn that around and ask “Why are so few women coming in to my degree?” then I have a useful question and, from various branches of research, the more rocks we turn over, the more we seem to find bias (conscious or otherwise) in both industry and academia that discourages women from participation in STEM.

A paper was recently published in the Proceedings of the National Academy of Sciences of the United States of America (PNAS, to its friends), entitled “Science faculty’s subtle gender biases favor male students”. (PNAS has an open access option but the key graphs and content are also covered in a Scientific American blog article.) The study was simple. Take a job application for  a lab manager position. Assign a name where half of the names are a recognisably male name, the other half are female. (The names John and Jennifer were chosen for this purpose as they had been pre-tested to be equivalent in terms of likability and recogniseability.) Get people to rate the application, including aspects like degree of mentoring offered and salary.

Let me summarise that: the name John or Jennifer is assigned to the same application materials. What we would expect, if there is no bias, is that we would see a similar ranking and equivalent salary offering. (All figures from the original paper, via the SciAm link.)

Oh. It appears that the mere presence of a woman’s name somehow altered reality so that an objective assessment of ability was warped through some sort of … I give up. Humour has escaped me. The name change has resulted in a systematic and significant downgrading of perceived ability. Let me get the next graph out of the way which is the salary offer.

And, equally mysteriously, having the name John is worth over $3,500 more than having the name Jennifer.

I should leap to note that it was both male and female scientists making this classification – which starts to lead us away from outright misogyny and towards ingrained and subtler prejudices. Did people resort to explicitly sexist reasoning to downgrade the candidates? No, they used sound reasoning to argue against the applicant’s competency. Except, of course, we draw back the curtain and suddenly reveal that our sound reasoning works one way when the applicant is a man, another if they are a woman.

Before you think “Oh, they must have targeted a given field, age group or gone after people who do or don’t have tenure”, the field, age and tenure status of the rating professors had no significant effect. This bias is pervasive among faculty, field, age, gender and status. The report also looked at mentoring and, regardless of the rater’s gender, they offered less mentoring to women.

Let’s be blunt. Study after study shows that if there are any gender differences at all, they are so small as to not even vaguely explain what we see in the representation of female students in certain fields and completely fails to explain their reduced progress in later life. However, the bias and stereotypes that people are operating under do not so much predict what will happen as shape what will happen. We are now aware of effects such as Stereotype Threat (Wiki link) that allows us to structure important situations in someone’s life so that the framing of the activity leaves them in a position where they reinforce the negative stereotype because of higher anxiety, relative to a non-stereotyped group. As an example, look at Osborne, Linking Stereotype Threat and Anxiety, where you can actually reduce the performance of girls on a maths test through reminding them that they are girls and that girls tend to do worse on test than boys. Osborne then compared this with a group where the difference was identified but a far more positive statement was made (the participants were told that despite the difference, there were situations where girls performed as well or better). The first scenario (girls do worse) was a high Stereotype Threat scenario (high ST), the second is low ST. Here’s the graph from Wikipedia that is a redrawing of the one in the paper that shows the results.

The effect of Stereotype Threat (ST) on math test scores for girls and boys. Data from Osborne (2007) (via Wikipedia)

That is the impact of an explicit stereotype in action – suddenly, when framed fairly and without an explicit stereotype or implicit bias, we see that people are far more similar than we thought. If anything, we have partially inverted the stereotype.

To return to my first paragraph, I said:

what has always angered me is the sheer weight of resistance to any research that (a) clearly demonstrates that bias exists to explain why this occurs, (b) identifies how performance can be manipulated through preconceptions and (c) requires people to consider that we are all more similar than current representation would indicate.

The PNAS paper, among others, clearly shows that the biasses exist. A simple name change is enough, as long as it’s a woman’s name. The demonstrated existence of stereotype threat shows us how performance can be manipulated through preconception. (And it’s important to note that stereotype threat is as powerful against minorities as woman – anyone who is part of a stereotype can be manipulated through their own increased or reduced anxiety.) So let me finally discuss the consideration of all of this and the title of this post.

I am expecting to get at least one person howling me down. Someone who will tear apart all of this because this cannot, possibly, under any circumstances be true. Someone who will start talking about our “African ancestors” to start arguing the Savanna-distribution of roles, as if our hominid predecessors ever had to apply to be a lab manager anywhere. Most of you, I hope, will read this and know all of this far too well. Some of you will reflect on this and, like me, examine yourself very carefully to find out if you have been using this bias or if you have been framing things, while trying to help, in a way that really didn’t help at all.

Some of you, who are my students, will read this and will see that research that you have done is reflected in these figures. Yes, we treat women differently and we appear, in these circumstances, to treat them less well. This does not, under any circumstances, mean that we have to accept this or, in any way, respect this as an established tradition or a desirable status quo. But the detection of an insidious and pervasive bias, that spans a community, shows us how hard my point (c) actually is.

We must first accept that there is a problem. There is a problem. Denying it will achieve nothing. Arguing minutiae will achieve nothing. We have to change the way that we react and be honest with ourselves that, sometimes, our treasured objectivity is actually nothing of the kind.

A Disappointing Reality

I read relatively regularly because aggregators help funnel information and their filter bias is not completely exclusive. An article that popped up recently dealt with the Kickstarter project of Anita Sarkeesian, who was asking for $6,000 to make a web series about “tropes vs women in video games”. There’s a New Statesman link here that you can follow for the whole unpleasant story but, assuming you’re in a hurry, let me summarise it for you.

  1. Blogger sees, from copious amounts of evidence, that video games seem to have trouble depicting women in reasonable and non-stereotyped ways.
  2. Blogger decides to set off Kickstarter to get money to produce a web series discussing this, money to cover research, playing more games and producing videos. (Blogger already has a track record in doing similar things for film.)
  3. Blogger becomes the target of attack, persistent, personal, vicious, violent, sick and twisted attacks from a skulking pit of suck that we call the Internet.

Anita Sarkeesian in her Kickstarter movie. (This image from

Here is a direct quote from Sarkeesian:

The intimidation and harassment effort has included a torrent of misogyny and hate speech on my YouTube video, repeated vandalizing of the Wikipedia page about me, organized efforts to flag my YouTube videos as “terrorism”, as well as many threatening messages sent through Twitter, Facebook, Kickstarter, email and my own website.  These messages and comments have included everything from the typical sandwich and kitchen “jokes” to threats of violence, death, sexual assault and rape.  All that plus an organized attempt to report this project to Kickstarter and get it banned or defunded.

You know what makes my heart sink? “The typical <x> jokes” because, of course, as a woman, I’m sorry, as a known woman on the Internet, she has seen and heard at least some of this before, just because she’s a woman. On her Wikipedia page, to quote the New Statesman article:

There are also references to Sarkeesian being “of Jewish descent”, an “entitled <racial epithet>” and having a “masters degree in Whining” (because why stick to one prejudice, when you can have them all?)

I can’t give you any more quotes because I try to keep this blog generally readable and there’s not much more I can say without having to ‘Adult rate’ this post.

Last year I attended a public seminar given by Professor Caroll Seron, who was a visiting international scholar in sociology and law at Flinders University, usually at UC Irvine, with talk entitled “The Changing Landscape of Women in the Professions: Why women study law and not engineering”. I went along, as an educator in Science, Technology, Engineering and Mathematics (STEM), I’m always looking for insight into why our female enrolments are so low and how we can improve them. What was most depressing about Professor Seron’s talk was that young women have similar reasons for going into engineering, they tend to do better financially but they tend to get relegated to gender roles once they go into work experience or work place environments, and then they leave. That is, a big group of mostly men will get the women to do what they think women should be doing, rather than letting them practice as engineers with their male counterparts.

It should come as no surprise that if you run a two-speed environment, or a free/constrained partitioning, the people that you are excluding will get the message and then they’ll leave. Which leaves fewer women in engineering, which gives us the same ‘women’s work’ nonsense workplaces.

So, much as I would like to think that it’s only the mindless Internet trolls that would act in such an obvious way, Professor Seron’s work suggests that the insidious attack on the validity of women in certain parts of the workplace is happening everywhere, every day. Until we address it, until we fix our culture, until we recognise that professional qualifications represent a capacity to do a job, regardless of which genitals we have, then what happened to Anita Sarkeesian is just a more obvious and, in some horrific ways, more honest account of how women are thought of every day, if they have the audacity to enter a ‘male sphere’.

Someone asked me for a name for a metadata repository today – for research and education. I suggested Hypatia. 2000 years and we haven’t got this rubbish sorted out yet? Seriously? Let’s strive for better.