Time Banking: Foresightedness and Reward

Posted: June 23, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , | 2 Comments

You may have noticed that I’ve stopped numbering the time banking posts – you may not have noticed that they were numbered in the first place! The reason is fairly simple and revolves around the fact that the numbers are actually meaningless. It’s not as if I have a huge plan of final sequence of the time banking posts. I do have a general idea but the order can change as one idea or another takes me and I feel that numbering them makes it look as if there is some grand sequence.

There isn’t. That’s why they all tend to have subtitles after them so that they can be identified and classified in a cognitive sequence. So, why am I telling you this? I’m telling you this so that you don’t expect “Time Banking 13” to be something special, or (please, no) “Time Banking 100” to herald the apocalypse.

The Druids invented time banking but could never find a sufficiently good Oracle to make it work. The Greeks had the Oracle but not the bank. This is why the Romans conquered everywhere. True story!

If I’m going to require students to self-regulate then, whether through operant or phenomenological mechanisms, the outcomes that they receive are going to have to be shaped to guide the student towards a self-regulating model. In simple terms, they should never feel that they have wasted their time, that they are under-appreciated or that they have been stupid to follow a certain path.

In particular, if we’re looking at time management, then we have to ensure that time spent in advance is never considered to be wasted time. What does that mean to me as a teacher, if I set an assignment in advance and students put work towards it – I can’t change the assignment arbitrarily. This is one of the core design considerations for time banking: if deadlines are seen as arbitrary (and extending them in case of power failures or class-wide lack of submission can show how arbitrary they are) then we allow the students to make movement around the original deadlines, in a way that gives them control without giving us too much extra work. If I want my students to commit to planning ahead and doing work before the due date then some heavy requirements fall on me:

  1. I have to provide the assignment work ahead of schedule and, preferably, for the entire course at the start of the semester.
  2. The assignments stay the same throughout that time. No last minute changes or substitutions.
  3. The oracle is tied to the assignment and is equally reliable.

This requires a great deal of forward planning and testing but, more importantly, it requires a commitment from me. If I am asking my students to commit, I have to commit my time and planning and attention to detail to my students. It’s that simple. Nobody likes to feel like a schmuck. Like they invested time under false pretences. That they had worked on what they thought was a commitment but it turned out that someone just hadn’t really thought things through.

Wasting time and effort discourages people. It makes people disengage. It makes them less trustful of you as an educator. It makes them less likely to trust you in the future. It reduces their desire to participate. This is the antithesis of what I’m after with increasing self-regulation and motivation to achieve this, which I label under the banner of my ‘time banking’ project.

But, of course, it’s not as if we’re not already labouring under this commitment to our students, at least implicitly. If we don’t follow the three requirements above then, at some stage, students will waste effort and, believe me, they’re going to question what they’re doing, why they’re bothering, and some of them will drop out, drift away and be lost to us forever. Never thinking that you’ve wasted your time, never feeling like a schmuck, seeing your ideas realised, achieving goals: that’s how we reward students, that’s what can motivate students and that’s how we can move the on to higher levels of function and achievement.

 

Flow, Happiness and the Pursuit of Significance

Posted: June 22, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , | Leave a comment

I’ve just been reading Deirdre McCloskey’s article on “Happyism” in The New Republic. While there are a number of points I could pick at in the article, I question her specific example of statistical significance and I think she’s oversimplified a number of the philosophical points, there are a lot of interesting thoughts and arguments within the article.

One of my challenges in connecting with my students is that of making them understand what the benefit is to them of adopting, or accepting, suggestions from me as to how to become better as discipline practitioners, as students and, to some extent, as people. It would be nice if doing the right thing in this regard could give the students a tangible and measurable benefit that they could accumulate on some sort of meter – I have performed well, my “success” meter has gone up by three units. As McCloskey points out, this effectively requires us to have a meter for something that we could call happiness, but it is then tied directly to events that give us pleasure, rather than a sequence of events that could give us happiness. Workflows (chains of actions that lead to an eventual outcome) can be assessed for accuracy and then the outcome measured, but it is only when the workflow is complete that we can assess the ‘success’ of the workflow and then derive pleasure, and hence happiness, from the completion of the workflow. Yes, we can compose a workflow from sub-workflows but we will hit the same problem if we focus on an outcome-based model – at some stage, we are likely to be carrying out an action that can lead to an event from which we can derive a notion of success, but this requires us to be foresighted and see the events as a chain that results in this outcome.

And this is very hard to meter and display in a way that says anything other than “Keep going!” Unsurprisingly, this is not really the best way to provide useful feedback, reward or fodder for self-actualisation.

I have a standing joke that, as a runner, I go to a sports doctor because if I go to a General Practitioner and say “My leg hurts after I run”, the GP will just say “Stop running.” I am enough of a doctor to say that to myself – so I seek someone who is trained to deal with my specific problems and who can give me a range of feedback that may include “stop running” because my injuries are serious or chronic, but can provide me with far more useful information from which I can make an informed choice. The happiness meter must be able to work with workflow in some way that is useful – keep going is not enough. We therefore need to look at the happiness meter.

McCloskey identifies Bentham, founder of utilitarianism, as the original “pleasure meter” proponent and implicitly addressed the beneficial calculus as subverting our assessment of “happiness units” (utils) into a form that assumes that we can reasonably compare utils between different people and that we can assemble all of our life’s experiences in a meaningful way in terms of utils in the first place!

To address the issue of workflow itself, McCloskey refers to the work of Mihály Csíkszentmihályi on flow: “the absorption in a task just within our competence”. I have talked about this before, in terms of Vygotsky’s zone of proximal development and the use of a group to assist people who are just outside of the zone of flow. The string of activities can now be measured in terms of satisfaction or immersion, as well as the outcomes of this process. Of course, we have the outcomes of the process in terms of direct products and we have outcomes in terms of personal achievement at producing those products. Which of these go onto the until meter, given that they are utterly self-assessed, subjective and, arguably, orthogonal in some cases. (If you have ever done your best, been proud of what you did, but failed in your objective, you know what I’m talking about.)

My reading of McCloskey is probably a little generous because I find her overall argument appealing. I believe that her argument may be distilled are:

  • If we are going to measure, we must measure sensibly and be very clear in our context and the interpretation of significance.
  • If we are going to base any activity on our measurement, then the activity we create or change must be related to the field of measurement.

Looking at the student experience in this light, asking students if they are happy with something is, ultimately, a pointless activity unless I either provide well-defined training in my measurement system and scale, or I am looking for a measurement of better or worse. This is confounded by simple cognitive biasses including, but not limited to, the Hawthorne Effect and confirmation bias. However, measuring what my students are doing, as Csíkszentmihályi did in the flow experiments, will show me if they are so engaged with their activities that they are staying in the flow zone. Similarly, looking at participation and measuring outputs in collaborative activities where I would expect the zone of proximal development to be in effect is going to be far more revealing than asking students if they liked something or not.

As McCloskey discusses, there is a point at which we don’t seem to get any happier but it is very hard to tell if this is a fault in our measurement and our presumption of a three-point non-interval scale and it then often degenerates into a form of intellectual snobbery that, unsurprisingly, favours the elites who will be studying the non-elites. (As an aside, I learnt a new word. Clerisy: “A distinct class of learned or literary people” If you’re going to talk about the literate elites, it’s nice to have a single word to do so!) In student terms, does this mean that there is a point at which even the most keen of our best and brightest will not try some of our new approaches? The question, of course, is whether the pursuit of happiness is paralleling the quest for knowledge, or whether this is all one long endured workflow that results in a pleasure quantum labelled ‘graduation’.

As I said, I found it to be an interesting and thoughtful piece, despite some problems and I recommend it to you, even if we must then start an large debate in the comments on how much I misled you!

Time Banking: Foundations

Posted: June 21, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , | Leave a comment

Short post today because I’ve spent so much time looking at research and fixing papers and catching up on things that I haven’t left myself much time to blog. Sorry about that! Today’s post is talking about one of the most vital aspects of time banking and one that I’ve been working on slightly under the radar – the theoretical underpinnings based on work in education, psychology and economics.

Foundations: More complex than they first appear.

Today we’ve been looking at key papers in educational psychology on motivation – but the one that stood out today was Zimmerman (90), “Self-regulated learning and academic achievement: An overview.” in Educational Psychologist, 25.  I want my students to become their own time managers but that’s really just a facet of self-regulation. It’s important to place all of this “let’s get rubber with time” into context and build on the good science that has gone before. I want my students to have the will to learn and practice, and the skill to do so competently – one without the other is no good to me.

This is, of course, just one of the aspects that we have to look at. Do I even know how I’m planning to address the students? Within an operant framework of punishment and reward or a phenomenological framework of self-esteem? How am I expecting them to think? These seem like rather theoretical matters but I need to know how existing timeliness issues are being perceived. If students think that they’re working in a reward/punishment framework then my solution has to take that into account. Of course, this takes us well into the world of surveying and qualitative analysis, but to design this survey we need sound theory and good hypotheses so that we can start in the ballpark of the right answer and iteratively improve.

We’re looking at motivation as the key driver here. Yes, we’re interested in student resilience and performance, but it’s the motivation to move to self-regulation that is what we’re trying to maximise. Today’s readings and sketching will be just one day out of many more to come as we further refine our search from the current broader fan to a more concentrated beam.

What of the economic factors? There is no doubt that the time bank forms a primitive economy out of ‘hours’ of a student’s time but it’s one where the budget doesn’t have to balance across the class, just across an individual student. This makes things easier to an extent as I don’t have to consider a multi-agent market beyond two people: the student and me. However, the student still has private information from me, the quality, progress and provenance of their work, and I have private information from them, in terms of the final mark. Can I make the system strategy proof, where students have no incentive to lie about how much work they’ve done or don’t try to present their private information in a way that is inherently non-truthful? Can I also produce a system where I don’t overly manipulate the system through the construction of the oracle or my support mechanisms? There’s a lot of great work out there on markets and economies so I have a great deal of reading to do here as well.

So, short post – but a long and fascinating day.

Transparency: Our universal requirement

Posted: June 20, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , | 1 Comment

There’s been a lot of discussion recently on the removal of Teresa Sullivan as the President of the University of Virginia. You can read about it here or Mark’s excellent summary and commentary here, for a couple of summaries, or just search for more details because there is certainly no shortage of information. There are many theories as to why it may have happened but, what is completely clear from almost every source, Teresa Sullivan had managed to make impressive levels of positive change happen and with the agreement and support of a large percentage of her faculty and administrators.

Of course, all of us want to know the real reasons ‘why’ behind this decision because if an apparently successful President can get fired there must be a good reason. Right?

An unbiassed lens. Value this – they’re rare.

Despite wanting to speak to the Board of Visitors in open session, Sullivan has only been offered a closed session and, from what both she and the board have released, we have no real information to go on.

This is a catastrophic failure of transparency – one of the qualities that we (should) cherish as educators because it is the core of our objectivity, our ability to replicate results and of a fair system. At the start of every semester, I tell the students how they can earn their marks, what I expect, what their deadlines are, what the penalties are, where they can find our policies, what these policies mean to them and, believe me, I consider myself as bound to these statements as I do to my marriage vows. (And I am exceedingly married.) If I don’t tell my students what they need to do, then they have a black box model – they try inputs, see outputs and try to map one to the other. I don’t want them to waste time on this because I want them to learn the important stuff. By being open and transparent in my approach to teaching, we focus on the right things and work out how to do what I want, rather than guessing (and probably getting it wrong).

We can talk about the influence of boards, or unpopular decisions, as much as we want but the issue here is that the next President of UVa had better work out pretty quickly what the secret transformation is concealed inside that Black Box or they will join Teresa Sullivan on the outside. How can you attract someone with the values that you should expect to have in this position, if you can’t even tell the world how they didn’t meet a secret set of rules, apparently put in place by a small group of people.

Telling people what they need to do, giving them guidance on how to do it, being open and transparent about what these requirements are, for all comers and at all times, are the cornerstones of a fair, balanced and modern education system.

It’s a shame that the Board of Visitors at University of Virginia appear to have forgotten this.

Your love is like bad measurement.

Posted: June 19, 2012 | Author: | Filed under: Education, Opinion | Tags: , , , , , , , , , , , , , | Leave a comment

(This is my 200th post. I’ve allowed myself a little more latitude on the opinionated scale. Educational content is still present but you may find some of the content slightly more confronting than usual. I’ve also allowed myself an awful pun in the title.)

People like numbers. They like solid figures, percentages, clear statements and certainty. It’s a great shame that mis-measurement is so easy to do, when you search for these figures, and so much a part of our lives. Today, I’m going to discuss precision and recall, because I eventually want to talk about bad measurement. It’s very easy to get measurement wrong but, even when it’s conducted correctly, the way that we measure or the reasons that we have for measuring can make even the most precise and delicate measurements useless to us for an objective scientific purpose. This is still bad measurement.

I’m going to give you a big bag of stones. Some of the stones have diamonds hidden inside them. Some of the stones are red on the outside. Let’s say that you decide that you are going to assume that all stones that have been coloured red contain diamonds. You pull out all of the red stones, but what you actually want is diamonds. The number of red stones are referred to as the number of retrieved instances – the things that you have selected out of that original bag of stones. Now, you get to crack them open and find out how many of them have diamonds. Let’s say you have R red stones and D1 diamonds that you found once you opened up the red stones. The precision is the fraction D1/R: what percentage of the stones that you selected (Red) were actually the ones that you wanted (Diamonds). Now let’s say that there are D2 diamonds (where D2 is greater than or equal to zero) left back in the bag. The total number of diamonds in that original bag was D1+D2, right? The recall is the fraction of the total number of things that you wanted (Diamonds, given by D1+D2) that you actually got (Diamonds that were also painted Red, which is D1). So this fraction is D1/(D1+D2),the number you got divided by the number that there were there for you to actually get.

Sorry, Logan5, your time is up.

If I don’t have any other mechanism that I can rely upon for picking diamonds out of the bag (assuming no-one has conveniently painted them red), and I want all of the diamonds, then I need to take all of them out. This will give me a recall of 100% (D2 will be 0 as there will be nothing left in the bag and the fraction will be D1/D1). Hooray! I have all of the diamonds! There’s only one problem – there are still only so many diamonds in that bag and (maybe) a lot more stones, so my precision may be terrible. More importantly, my technique sucks (to use an official term) and I have no actual way of finding diamonds. I just happen to have used a mechanism that gets me everything so it must, as a side effect, get me all of the diamonds. I haven’t actually done anything except move everything from one bag to another.

One of the things about selection mechanisms is that people often seem happy to talk about one side of the precision/recall issue. “I got all of them” is fine but not if you haven’t actually reduced your problem at all. “All the ones I picked were the right ones” sounds fantastic until you realise that you don’t know how many were left behind that were also the ones that you wanted. If we can specify solutions (or selection strategies) in terms of their precision and their recall, we can start to compare them. This is an example of how something that appears to be straightforward can actually be a bad measurement – leave out one side of precision or recall and you have no real way of assessing the utility of what it is that you’re talking about, despite having some concrete numbers to fall back on.

You may have heard this expressed in another way. Let’s assume that you can have a mechanism for determining if people are innocent or guilty of a crime. If it was a perfect mechanism, then only innocent people would go free and only guilt people would go to jail. (Let’s assume it’s a crime for which a custodial sentence is appropriate.) Now, let’s assume that we don’t have a perfect mechanism so we have to make a choice – either we set up our system so that no innocent person goes to jail, or we set up our system so that no guilty person is set free. It’s fairly easy to see how our interpretation of the presumption of innocence, the notion of reasonable doubt and even evidentiary laws would be constructed in different ways under either of these assumptions. Ultimately, this is an issue of precision and recall and by understanding these concepts we can define what we are actually trying to achieve. (The foundation of most modern law is that innocent people don’t go to jail. A number of changes in certain areas are moving more towards a ‘no one who may be guilty of crimes of a certain type will escape us’ model and, unsurprisingly, this is causing problems due to inconsistent applications of our simple definitions from above.)

The reason that I brought all of this up was to talk about bad measurement, where we measure things and then over-interpet (torture the data) or over-assume (the only way that this could have happened was…) or over-claim (this always means that). It is possible to have a precise measurement of something and still be completely wrong about why it is occurring. It is possible that all of the data that we collect is the wrong data – collected because our fundamental hypothesis is in error. Data gives us information but our interpretative framework is crucial in determining what use we can make of this data. I talked about this yesterday and stressed the importance of having enough data, but you really have to know what your data means in order to be sure that you can even start to understand what ‘enough data’ means.

One example is the miasma theory of disease – the idea that bad smells caused disease outbreaks. You could construct a gadget that measured smells and then, say in 18th Century England, correlate this with disease outbreaks – and get quite a good correlation. This is still a bad measurement because we’re actually measuring two effects, rather than a cause (dead mammals introducing decaying matter/faecal bacteria etc into water or food pathways) and the effects (smell of decomposition, and diseases like cholera, E. Coli contamination, and so on). We can collect as much ‘smell’ data as we like, but we’re unlikely to learn much more because any techniques that focus on the smell and reducing it will only work if we do things like remove the odiferous elements, rather than just using scent bags and pomanders to mask the smell.

To look at another example, let’s talk about the number of women in Computer Science at the tertiary level. In Australia, it’s certainly pretty low in many Universities. Now, we can measure the number of women in Computer Science and we can tell you exactly how many are in a given class, what their average marks are, and all sorts of statistical data about them. The risk here is that, from the measurements alone, I may have no real idea of what has led to the low enrolments for women in Computer Science.

I have heard, far too many times, that there are too few women in Computer Science because women are ‘not good at maths/computer science/non-humanities courses’ and, as I also mentioned recently when talking about the work of Professor Seron, this doesn’t appear to the reason at all. When we look at female academic performance, reasons for doing the degree and try to separate men and women, we don’t get the clear separation that would support this assertion. In fact, what we see is that the representation of women in Computer Science is far lower than we would expect to see from the (marginally small) difference that does appear at the very top end of the data. Interesting. Once we actually start measuring, we have to question our hypothesis.

Or we can abandon our principles and our heritage as scientists and just measure something else that agrees with us.

You don’t have to get your measurement methods wrong to conduct bad measurement. You can also be looking for the wrong thing and measure it precisely, because you are attempting to find data that verifies your hypothesis, but rather than being open to change if you find contradiction, you can twist your measurements to meet your hypothesis, you can only collect the data that supports your assumptions and you can over-generalise from a small scale, or from another area.

When we look at the data, and survey people to find out the reasons behind the numbers, we reduce the risk that our measurements don’t actually serve a clear scientific purpose. For example, and as I’ve mentioned before, the reason that there are too few women studying Computer Science appears to be unpleasantly circular and relates to the fact that there are too few women in the discipline over all, reducing support in the workplace, development opportunities and producing a two-speed system that excludes the ‘newcomers’. Sorry, Ada and Grace (to name but two), it turns out that we seem to have very short memories.

Too often, measurement is conducted to reassure ourselves of our confirmed and immutable beliefs – people measure to say that ‘this race of people are all criminals/cheats/have this characteristic’ or ‘women cannot carry out this action’ or ‘poor people always perform this set of actions’ without necessarily asking themselves if the measurement is going to be useful, or if this is useful pursuit as part of something larger. Measuring in a way that really doesn’t provide any more information is just an empty and disingenuous confirmation. This is forcing people into a ghetto, then declaring that “all of these people live in a ghetto so they must like living in a ghetto”.

Presented a certain way, poor and misleading measurement can only lead to questionable interpretation, usually to serve a less than noble and utterly non-scientific goal. It’s bad enough when the media does it but it’s terrible when scientists, educators and academics do it.

Without valid data, collected on the understanding that a world-changing piece of data could actually change our data, all our work is worthless. A world based on data collection purely for the sake of propping up, with no possibility of discovery and adaptation, is a world of very bad measurement.

The Many Types of Failure: What Does Zero Mean When Nothing Is Handed Up?

Posted: June 18, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , , , , , , , , , | 3 Comments

You may have read about the Edmonton, Canada, teacher who expected to be sacked for handing out zeros. It’s been linked to sites as diverse as Metafilter, where a long and interesting debate ensued, and Cracked, where it was labelled one of the ongoing ‘pussifications’ of schools. (Seriously? I know you’re a humour site but was there some other way you could have put that? Very disappointed.)

Basically, the Edmonton Public School Board decided that, rather than just give a zero for a missed assignment, this would be used as a cue for follow-up work and additional classes at school or home. Their argument – you can’t mark work that hasn’t been submitted, let’s use this as a trigger to try and get submission, in case the source is external or behavioural. This, of course, puts the onus on the school to track the students, get the additional work completed, and then mark out of sequence. Lynden Dorval, the high school teacher who is at the centre of this, believe that there is too much manpower involved in doing this and that giving the student a zero forces them to come to you instead.

Some of you may never have seen one of these before. This is a zero, which is the lowest mark you can be awarded for any activity. (I hope!)

Now, of course, this has split people into two fairly neat camps – those who believe that Dorval is the “hero of zero” and those who can see the benefit of the approach, including taking into account that students still can fail if they don’t do enough work. (Where do I stand? I’d like to know a lot more than one news story before I ‘pick a side’.) I would note that a lot of tired argument and pejorative terminology has also come to the fore – you can read most of the buzzwords used against ‘progressives’ in this article, if you really want to. (I can probably summarise it for you but I wouldn’t do it objectively. This is just one example of those who are feting Dorval.)

Of course, rather than get into a heated debate where I really don’t have enough information to contribute, I’d rather talk about the basic concept – what exactly does a zero mean? If you hand something in and it meets none of my requirements, then a zero is the correct and obvious mark. But what happens if you don’t hand anything in?

With the marking approach that I practice and advertise, which uses time-based mark penalties for late submission, students are awarded marks for what they get right, rather than have marks deducted for what they do wrong. Under this scheme, “no submission” gives me nothing to mark, which means that I cannot give you any marks legitimately – so is this a straight-forward zero situation? The time penalties are in place as part of the professional skill requirements and are clearly advertised, and consistently policed. I note that I am still happy to give students the same level of feedback on late work, including their final mark without penalty, which meets all of the pedagogical requirements, but the time management issues can cost a student some, most or all of their marks. (Obviously, I’m actively working on improving engagement with time management through mechanisms that are not penalty based but that’s for other posts.)

As an aside, we have three distinct fail grades for courses at my University:

  • Withdraw Fail (WF), where a student has dropped the course but after the census date. They pay the money, it stays on their record, but as a WF.
  • Fail (F), student did something but not enough to pass.
  • Fail No Submission (FNS), student submitted no work for assessment throughout the course.

Interestingly, for my Uni, FNS has a numerical grade of 0, although this is not shown on the transcript. Zero, in the course sense, means that you did absolutely nothing. In many senses, this represents the nadir of student engagement, given that many courses have somewhere from 1-5, maybe even 10%, of marks available for very simple activities that require very little effort.

My biggest problem with late work, or no submission, is that one of the strongest messages I have from that enormous data corpus of student submission that I keep talking about is that starting a pattern of late or no submission is an excellent indicator of reduced overall performance and, with recent analysis, a sharply decreased likelihood of making it to third year (final year) in your college studies. So I really want students to hand something in – which brings me to the crux of the way that we deal with poor submission patterns.

Whichever approach I take should be the one that is most likely to bring students back into a regular submission pattern. 

If the Public School Board’s approach is increasing completion rates and this has a knock-on effect which increases completion rates in the future? Maybe it’s time to look at that resourcing profile and put the required money into this project. If it’s a transient peak that falls off because we’re just passing people who should be failing? Fuhgeddaboutit.

To quote Sherlock Holmes (Conan Doyle, naturally): 

It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts. (A Scandal in Bohemia)

“Data! Data! Data!” he cried impatiently. “I can’t make bricks without clay.” (The Adventure of the Copper Beeches)

It is very easy to take a side on this and it is very easy to see how both sides could have merit. The issue, however, is what each of these approaches actually does to encourage students to submit their assignment work in a more timely fashion. Experiments, experimental design, surveys, longitudinal analysis, data, data, data!

If I may end by waxing lyrical for a moment (and you will see why I stick to technical writing):

If zeroes make Heroes, then zeroes they must have! If nulls make for dulls, then we must seek other ways!

A Disappointing Reality

Posted: June 16, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , | 2 Comments

I read metafilter.com relatively regularly because aggregators help funnel information and their filter bias is not completely exclusive. An article that popped up recently dealt with the Kickstarter project of Anita Sarkeesian, who was asking for $6,000 to make a web series about “tropes vs women in video games”. There’s a New Statesman link here that you can follow for the whole unpleasant story but, assuming you’re in a hurry, let me summarise it for you.

  1. Blogger sees, from copious amounts of evidence, that video games seem to have trouble depicting women in reasonable and non-stereotyped ways.
  2. Blogger decides to set off Kickstarter to get money to produce a web series discussing this, money to cover research, playing more games and producing videos. (Blogger already has a track record in doing similar things for film.)
  3. Blogger becomes the target of attack, persistent, personal, vicious, violent, sick and twisted attacks from a skulking pit of suck that we call the Internet.

Anita Sarkeesian in her Kickstarter movie. (This image from newstatesmen.com.)

Here is a direct quote from Sarkeesian:

The intimidation and harassment effort has included a torrent of misogyny and hate speech on my YouTube video, repeated vandalizing of the Wikipedia page about me, organized efforts to flag my YouTube videos as “terrorism”, as well as many threatening messages sent through Twitter, Facebook, Kickstarter, email and my own website.  These messages and comments have included everything from the typical sandwich and kitchen “jokes” to threats of violence, death, sexual assault and rape.  All that plus an organized attempt to report this project to Kickstarter and get it banned or defunded.

You know what makes my heart sink? “The typical <x> jokes” because, of course, as a woman, I’m sorry, as a known woman on the Internet, she has seen and heard at least some of this before, just because she’s a woman. On her Wikipedia page, to quote the New Statesman article:

There are also references to Sarkeesian being “of Jewish descent”, an “entitled <racial epithet>” and having a “masters degree in Whining” (because why stick to one prejudice, when you can have them all?)

I can’t give you any more quotes because I try to keep this blog generally readable and there’s not much more I can say without having to ‘Adult rate’ this post.

Last year I attended a public seminar given by Professor Caroll Seron, who was a visiting international scholar in sociology and law at Flinders University, usually at UC Irvine, with talk entitled “The Changing Landscape of Women in the Professions: Why women study law and not engineering”. I went along, as an educator in Science, Technology, Engineering and Mathematics (STEM), I’m always looking for insight into why our female enrolments are so low and how we can improve them. What was most depressing about Professor Seron’s talk was that young women have similar reasons for going into engineering, they tend to do better financially but they tend to get relegated to gender roles once they go into work experience or work place environments, and then they leave. That is, a big group of mostly men will get the women to do what they think women should be doing, rather than letting them practice as engineers with their male counterparts.

It should come as no surprise that if you run a two-speed environment, or a free/constrained partitioning, the people that you are excluding will get the message and then they’ll leave. Which leaves fewer women in engineering, which gives us the same ‘women’s work’ nonsense workplaces.

So, much as I would like to think that it’s only the mindless Internet trolls that would act in such an obvious way, Professor Seron’s work suggests that the insidious attack on the validity of women in certain parts of the workplace is happening everywhere, every day. Until we address it, until we fix our culture, until we recognise that professional qualifications represent a capacity to do a job, regardless of which genitals we have, then what happened to Anita Sarkeesian is just a more obvious and, in some horrific ways, more honest account of how women are thought of every day, if they have the audacity to enter a ‘male sphere’.

Someone asked me for a name for a metadata repository today – for research and education. I suggested Hypatia. 2000 years and we haven’t got this rubbish sorted out yet? Seriously? Let’s strive for better.

Personal Reflection on Time Management: Why I Am a Bad Role Model.

Posted: June 15, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , | 2 Comments

Do any of you remember that scene from “Pretty Woman”, when Richard Gere’s character Edward asks his ex-girlfriend Susan if she had spent more time talking to his secretary than to him? Her reply was, simply, “she was one of my bridesmaids”. Any time in a relationship of any kind that you don’t meet the needs of the people in the relationship, it’s going to cause problems. (Not all problems have to end with you scaling a ladder to patch up your relationship with Julia Roberts but there will be problems.) Now, before anyone is wondering (or my friends are worried), my marriage is still great, I’m talking about my professional relationship with my students.

Ah, the old Time == Money equivalence, via an article on portfoliosc.blogspot.sg

While I’ve been critical of myself and my teaching this semester, where I’ve done a good job but not necessarily excelled across the board, I haven’t identified one of the greatest problems that has crept in – no-one in their right mind should be emulating what I laughingly refer to as my work ethic, time commitment or current pursuit of success. Right now, I am not a good role model for students. While I am still ethical, professional, knowledgable and I’m apparently doing a good job, I cannot present myself as a role model to my students because I am losing the time that I need to seize new opportunities, and to allow for the unrushed catch-up time with other people that is vital to doing a job such as mine and doing it well. At the moment, any student who comes to see me does so knowing that I will have 15-30 minutes, tops, and that I will then have to rush off, elsewhere, to go and do something else. It doesn’t matter when they come to see me – if I’m in at 7:30am it’s because the day is starting at 8. If I’m still there at 7pm at night it’s because I have to be in order to meet requirements for that day or the next. Let me give you an example from my lectures.

My Student Experience of Learning and Teaching results have come in and there have been a lot of warm and rewarding comments from my students, among a pleasing overall rating. But one of my students hit the nail on the head. “[Nick] always seems to have a lot to say and constantly looks at his watch. (I assume that it’s to keep within time constraints) the problem is that he feels like he’s rushing.”

Ouch. That’s far too true and, while only one student noted it, you bet every other student was watching me use my watch to check my time progress through a busy, informative but ultimately time constrained lecture and at least some of them thought “Hmm, I have a question but I don’t want to bother him.”

It’s my job to be bothered! It’s my job to answer questions! Right now, it’s pretty obvious that students are getting the vibe that I’m a good lecturer, I care about them, I’m working well to give them the right knowledge and they love the course that I built… but… they don’t want to bother me because I’m too busy. Because I look too busy.

Every student who comes to see does so in the one of the windows that I have in my day, often between meetings, my meetings back up on each other with monotonous regularity and, looking at my calendar for last, week, the total amount of time that was uncommitted prior to the week starting was…

75 minutes.

Including the fact that Tuesday started at 7:30am and went until 6:15pm.

Please believe me when I say that I’m not boasting – I’m not proud of this, I think it’s the sign of poor scheduling and workaholism. This should be read as what it is, a sign that I have let my responsibilities pile up in a way that means that I am running the risk of becoming a stereotypically “grumpy old Professor”, who is too busy to see students.

So when you read all of this stuff about Time Banking and think “Well, I guess I can see some of his point – for assignments…” I’m trying to work out how I can take the primary goal of time banking – to make people think about their time commitments in a way that allows them to approximate a manageable uniformity of effort across time to achieve good results – and to work out how I can think about my own time in the same way. How do I adjust my boundaries in time and renegotiate while providing my own oracle and incentives for change? If I can crack that, then solving the student problem should have been made much easier.

How do I become the kind of person that I would want my students to be? Right now, it requires me to think about my commitments and my time, to treat my time as a scarce and precious commodity, but in a way that allows me to do all of the things I need to do in my job and all of the things that I love to do in my job, yet still have the time to sit around, grab a slow coffee, make a lunch booking with someone with less than a month’s notice and to get my breathing room back.

I have one of the best jobs in the world but the way I’m doing it is probably unsustainable and it’s not really in the spirit of the job that I want to do. It’s more than just me, too. I need to be seen to be approachable and to do that I have to actually be approachable, which means finding a way that makes me worthy of being a good overall role model again.

Time Banking IV: The Role of the Oracle

Posted: June 14, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , | 1 Comment

I’ve never really gone into much detail on how I would make a system like Time Banking work. If a student can meet my requirements and submit their work early then, obviously, I have to provide some sort of mechanism that allows the students to know that my requirements have been met. The first option is that I mark everything as it comes in and then give the student their mark, allowing them to resubmit until they get 100%.

That’s not going to work, unfortunately, as, like so many people, I don’t have the time to mark every student’s assignment over and over again. I wait until all assignments have been submitted, review them as a group, mark them as a group and get the best use out of staying in the same contextual framework and working on the same assignments. If I took a piecemeal approach to marking, it would take me longer and, especially if the student still had some work to do, I could end up marking the same assignment 3,4, however many times and multiplying my load in an unsupportable way.

Now, of course I can come up with simple measures that the students can check for themselves. Of course, the problem we have here is setting something that a student can mis-measure as easily as they measure. If I say  “You must have at least three pages for an essay” I risk getting three pages of rubbish or triple spaced 18 point print. It’s the same for any measure of quantity (number of words, number of citations, length of comments and so on) instead of quality. The problem is, once again, that if the students were capable of determining the quality of their own work and determining the effort and quality required to pass, they wouldn’t need time banking because their processes are already mature!

So I’m looking for an indicator of quality that a student can use to check their work and that costs me only (at most) a small amount of effort. In Computer Science, I can ask the students to test their work against a set of known inputs and then running their program to see what outputs we get. There is then the immediate problem of students hacking their code and just throwing it against the testing suite to see if they can fluke their way to a solution. So, even when I have an idea of how my oracle, my measure of meeting requirements, is going to work, there are still many implementation details to sort out.

Fortunately, to help me, I have over five years worth of student data through our automated assignment submission gateway where some assignments have an oracle, some have a detailed oracle, some have a limited oracle and some just say “Thanks for your submission.” The next stage in the design of the oracle is to go back and to see what impact the indications of progress and completeness had on the students. Most importantly, for me, is the indication of how many marks a student had to get in order to stop trying to make fresh submissions. If before the due date, did they always strive for 100%? If late, did they tend to stop at more than 50% of achieved marks, or more than 40% in the case of trying to avoid receiving a failing grade based on low assignment submission?

Are there significant and measurable differences between assignments with an oracle and those that have none (or a ‘stub’, so to speak)? I know what many people expect to find in the data, but now I have the data and I can go and interrogate that!

Every time that I have questions like this about the implementation, I have a large advantage in that I already have a large control body of data, before any attempts were made to introduce time banking. I can look at this to see what student behaviour is like and try to extract these elements and use them to assist students in smoothing out their application of effort and develop more mature time management approaches.

Now to see what the data actually says – I hope to post more on this particular aspect in the next week or so.

Time Banking III: Cheating and Meta-Cheating

Posted: June 13, 2012 | Author: | Filed under: Education | Tags: , , , , , , , , , , , , , | Leave a comment

One of the problems with setting up any new marking system is that, especially when you’re trying to do something a bit out of the ordinary, you have to make sure that you don’t produce a system that can be gamed or manipulated to let people get an unfair advantage. (Students are very resourceful when it comes to this – anyone who has received a mysteriously corrupted Word document of precisely the right length and with enough relevant strings to look convincing, on more than one occasion from the same student and they then are able to hand up a working one the next Monday, knows exactly what I’m talking about.)

As part of my design, I have to be clear to the students what I do and don’t consider to be reasonable behaviour (returning to Dickinson and McIntyre, I need to be clear in my origination and leadership role). Let me illustrate this with an anecdote from decades ago.

In the early 90s, I helped to write and run a number of Multi User Dungeons (MUDs) – the text-based fore-runners of the Massively Multiplayer On-line Role Playing Games, such as World of Warcraft. The games had very little graphical complexity and we spent most of our time writing the code that drove things like hitting orcs with swords or allowing people to cast spells. Because of the many interactions between the software components in the code, it was possible for unexpected things to happen – not just bugs where code stopped working but strange ‘features’ where things kept working but in an odd way. I knew a guy, let’s call him K, who was a long-term player of MUDs. If the MUD was any good, he’d not only played it, he’d effectively beaten it. He knew every trick, every lurk, the best way to attack a monster but, more interestingly, he had a nose for spotting errors in the code and taking advantage of them. One time, in a game we were writing, we spotted K walking around with something like 20-30 ’empty’ water bottles on him. (As game writers, wizards, we could examine any object in the game, which included seeing what players were carrying.)

A bit like this, but all on one person’s shoulders and no wheels.

This was weird. Players had a limited amount of stuff that they could carry, and K should have had no reason to carry those bottles. When we examined him, we discovered that we’d made an error in the code so that, when you drank from a bottle and emptied it, the bottle ended up weighing LESS THAN NOTHING. (It was a text game and our testing wasn’t always fantastic – I learnt!) So K was carrying around the in-game equivalent of helium balloons that allowed him to carry a lot more than he usually would.

Of course, once we detected it, we fixed the code and K stopped carrying so many empty bottles. (Although, I have no doubt that he personally checked each and every container we put into the game from that point on to see if could get it to happen again.) Did we punish him? No. We knew that K would need some ‘flexibility’ in his exploration of the game, knowing that he would press hard against the rubber sheet to see how much he could bend reality, but also knowing that he would spot problems that would take us weeks or months of time to find on our own. We took him into our new and vulnerable game knowing that if he tried to actually break or crash the game, or share the things he’d learned, we’d close off his access. And he knew that too.

Had I placed a limit in play that said “Cheating detected = Immediate Booting from the game”, K would have left immediately. I suspect he would have taken umbrage at the term ‘cheating’, as he generally saw it as “this is the way the world works – it’s not my fault that your world behaves strangely”. (Let’s not get into this debate right now, we’re not in the educational plagiarism/cheating space right now.)

We gave K some exploration space, more than many people would feel comfortable with, but we maintained some hard pragmatic limits to keep things working and we maintained the authority required to exercise these limits. In return, K helped us although, of course, he played for the fun of the game and, I suspect, the joy of discovering crazy bugs. However, overall, this approach saved us effort and load, and allowed us to focus on other things with our limited resources. Of course, to make this work required careful orientation and monitoring on our behalf. Nothing, after all, comes for free.

If I’d asked K to fill out forms describing the bugs he’d found, he’d never have done it. If I’d had to write detailed test documents for him, I wouldn’t have had time to do anything else. But it also illustrates something that I have to be very cautious of, which I’ve embodied as the ‘no cheating/gaming’ guideline for Time Banking. One of the problems with students at early development stages is that they can assume that their approach is right, or even assert that their approach is the correct one, when it is not aligned with our goals or intentions at all. Therefore, we have to be clear on the goals and open about our intentions. Given that the goal of Time Banking is to develop mature approach to time management, using the team approach I’ve already discussed, I need to be very clear in the guidance I give to students.

However, I also need to be realistic. There is a possibility that, especially on the first run, I introduce a feature in either the design or the supporting system that allows students to do something that they shouldn’t. So here’s my plan for dealing with this:

  1. There is a clear no-cheating policy. Get caught doing anything that tries to subvert the system or get you more hours in any other way than submitting your own work early and it’s treated as a cheating incident and you’re removed from the time bank.
  2. Reporting a significant fault in the system, that you have either deduced, or observed, is worth 24 hours of time to the first person who reports it. (Significant needs definition but it’s more than typos.)

I need the stick. Some of my students need to know that the stick is there, even if the stick is never needed, but I really can’t stand the stick. I have always preferred the carrot. Find me a problem and you get an automatic one-day extension, good for any assignment in the bank. Heck, I could even see my way clear to making this ‘liftable’ hours – 24 hours you can hand on to a friend if you want. If part of your team thinking extends to other people and, instead of a gifted student handing out their assignment, they hand out some hours, I have no problem with that. (Mr Pragmatism, of course, places a limit on the number of unearned hours you can do this with, from the recipient’s, not the donor’s perspective. If I want behaviour to change, then people have to act to change themselves.)

My design needs to keep the load down, the rewards up but, most importantly, the rewards have to move the students towards the same goals as the primary activity or I will cause off-task optimisation and I really don’t want to do that.

I’m working on a discussion document to go out to people who think this is a great idea, a terrible idea, the worst idea ever, something that they’d like to do, so that I can bring all of the thoughts back together and, as a group of people dedicated to education, come up with something that might be useful – OR, and it’s a big or, come up with the dragon slaying notion that kills time banking stone dead and provides the sound theoretical and evidence-based support as to why we must and always should use deadlines. I’m prepared for one, the other, both or neither to be true, along with degrees along the axis.

 

← Older Entries
Newer Entries →