I was recently at a conference-like event where someone stood up and talked about video lectures. And these lectures were about 40 minutes long.
Over several million viewing sessions, EdX have clearly shown that watchable video length tops out at just over 6 minutes. And that’s the same for certificate-earning students and the people who have enrolled for fun. At 9 minutes, students are watching for fewer than 6 minutes. At the 40 minute mark, it’s 3-4 minutes.
I raised this point to the speaker because I like the idea that, if we do on-line it should be good on-line, and I got a response that was basically “Yes, I know that but I think the students should be watching these anyway.” Um. Six minutes is the limit but, hey, students, sit there for this time anyway.
We have never been able to unobtrusively measure certain student activities as well as we can today. I admit that it’s hard to measure actual attention by looking at video activity time but it’s also hard to measure activity by watching students in a lecture theatre. When we add clickers to measure lecture activity, we change the activity and, unsurprisingly, clicker-based assessment of lecture attentiveness gives us different numbers to observation of note-taking. We can monitor video activity by watching what the student actually does and pausing/stopping a video is a very clear signal of “I’m done”. The fact that students are less likely to watch as far on longer videos is a pretty interesting one because it implies that students will hold on for a while if the end is in sight.
In a lecture, we think students fade after about 15-20 minutes but, because of physical implications, peer pressure, politeness and inertia, we don’t know how many students have silently switched off before that because very few will just get up and leave. That 6 minute figure may be the true measure of how long a human will remain engaged in this kind of task when there is no active component and we are asking them to process or retain complex cognitive content. (Speculation, here, as I’m still reading into one of these areas but you see where I’m going.) We know that cognitive load is a complicated thing and that identifying subgoals of learning makes a difference in cognitive load (Morrison, Margulieux, Guzdial) but, in so many cases, this isn’t what is happening in those long videos, they’re just someone talking with loose scaffolding. Having designed courses with short videos I can tell you that it forces you, as the designer and teacher, to focus on exactly what you want to say and it really helps in making your points, clearly. Implicit sub-goal labelling, anyone? (I can hear Briana and Mark warming up their keyboards!)
If you want to make your videos 40 minutes long, I can’t stop you. But I can tell you that everything I know tells me that you have set your materials up for another hominid species because you’re not providing something that’s likely to be effective for current humans.
SIGCSE 2014: Collecting and Analysing Student Data 1, “AP CS Data”, Thursday 3:15 – 5:00pm (#SIGCSE2014)Posted: March 7, 2014
The first paper, “Measuring Demographics and Performance in Computer Science Education at a Nationwide Scale using AP CS Data”, from Barbara Ericson and Mark Guzdial, has been mentioned in these hallowed pages before, as well as Mark’s blog (understandably). Barb’s media commitments have (fortunately) slowed down btu it was great to see so many people and the media taking the issue of under-representation in AP CS seriously for a change. Mark presented and introduced the Advanced Placement CS program which is the only nationwide measure of CS education in the US. This allows us tto use the AP CS to compare with other AP exams, find who is taking the AP CS exams and how well do they perform. Looking longitudinally, how has this changed and what influences exam-taking? (There’s been an injection of funds into some states – did this work?)
The AP are exams you can take while in secondary school that gives you college credit or placement in college (similar to the A levels, as Mark put it). There’s an audit process of the materials before a school can get accreditation. The AP exam is scored 1-5, where 3 is passing. The overall stats are a bit worrying, when Back, Hispanic and female students are grossly under-represented. Look at AP Calculus and this really isn’t as true for female students and there is better representation for Black and Hispanic students. (CS has 4% Black and 7.7% Hispanic students when America’s population is 13.1% Black students and 16.9% Hispanic) The pass rates for AP Calculus are about the same as for AP CS so what’s happening?
Looking at a (very cool) diagram, you see that AP overall is female heavy – CS is a teeny, tiny dot and is the most male dominated area, and 1/10th the size of calculus. Comparing AP CS to the others. there has been steady growth since 1997 in Calculus, Biology, Stats, Physics, Chem and Env Science AP exams – but CS is a flat, sunken pancake that hasn’t grown much at all. Mark then analysed the data by states, counting the number of states in each category along features such as ‘schools passing audit/10K pop’, #exams/pop and % passing exams. Mark then moved onto diversity data: female, Black and Hispanic test takers. It’s worth noting that Jill Pala made the difference to the entire state she taught in, raising the number of women. Go, Jill! (And she asked a really good question in my talk, thanks again, Jill!)
How has this changed overtime? California and Maryland have really rapid growth in exam takers over the last 6 years, with NSF involvement. But Michigan and Indiana have seen much less improvement. In Georgia, there’s overall improvement, but mostly women and Hispanic students, but not as much for Black students. The NSF funding appears to have paid off, GA and MA have improved over the last 6 years but female test takers have still not exceeded 25% in the last 6 years.
Why? What influences exam taking?
- The wealth in the state influences number of schools passing audit
- Most of the variance in the states comes from under-representation in certain groups.
It’s hard to add wealth but if you want more exam takers, increase your under-represtentation group representation! That’s the difference between the states.
Conclusions? It’s hard to compare things most of the time and the AP CS is the best national pulse we have right now. Efforts to improve are having an effect but wealth matters, as in the rest of education.
All delivered at a VERY high speed but completely comprehensible – I think Mark was trying to see how fast I can blog!
Mark Guzdial has put out some excellent posts recently on Barbara Ericson’s ongoing work on analysing AP CS exam attempts and results across the US. Unsurprisingly, to those of us who see the classrooms on a day-to-day basis, women are grossly underrepresented. In this interview, Barbara is quoted:
Barbara Ericson, director of computing outreach at Georgia Tech, has made a startling claim. She said not one female student in three states – Mississippi, Montana and Wyoming — took the Advanced Placement exam in computer science last year.
Ericson appeared on Weekend Express to discuss the gender gap and explains why more women aren’t interested in computer science.
Now, I’m not going to rehash all of these posts but I did want to pick on one blogger who took the AP data and then, as far as I’m concerned, not only got it wrong by making some fundamental interpretational errors but managed to do so in a way that so heavily reeked of privilege that I’m going to call it out.
I hesitate to link to the article on the Huffington Post but it’s only fair that you should read it to see what you think, even though it will generate traffic. The article is called “Memo to Chicken Little: Female Scientists Do Roam Among Us, and Gasp! Some Even Wear Lipstick”. So before we’ve even started, we’ve got one good stereotype going in the title.
Look, I’m not planning to drag apart the whole article but I will pick on one point that the author makes because it really irritates me. Here’s the paragraph:
As a woman who likes science as a bystander but chose not to pursue it professionally, I’ve got a couple of problems with all this handwringing. Mostly, well-intentioned as it is, it implies that women need “help” choosing a field of study. High school girls are exposed to exactly the same science and math courses they need to graduate as boys are, but in the eyes of the handwringers, girls are either too shallow or simple to choose for themselves, or need to be socially engineered into the correct balance of male vs. female, regardless of their choices. I appreciate your concern, but frankly, it’s pretty demeaning.
Frankly, I’ve never seen a more disingenuous interpretation of attempts to undo and reverse the systematic anti-female bias that is built into our culture. I’ve never seen anyone who is trying to address this problem directly or indirectly label girls as shallow or too simple to choose – this is a very unpleasant strawman, constructed to make those of us who are trying to address a bias look like we’re the ones with the attitude problem. We don’t need to socially engineer girls into the correct balance, we need to engineer society to restore the balance and articles like this, which make it appear that women are deliberately choosing to avoid STEM, are unwelcome, unnecessary and unfair to the many young women who are being told that the way that our society works is the way that it should work.
Need I remind people of stereotype threat? The PNAS study that shows that women are as automatically likely to harshly judge women and lessen their rewards as their male colleagues? Looking at the AP attendance and performance doesn’t show equality, it shows the outcome of a systematically biased system.
To say that “High school girls are exposed to exactly the same science and math courses they need to graduate as boys are” is a difficult statement. Yes, women rack up roughly the same number of course credits but on the critical measurement of whether they choose to go on and pursue a profession? No, something breaks here. The AP test is a great measure because it is an Advanced Placement exam and your intention is to use this to go further. Is there clear evidence of far fewer women, as a percentage, going on from high school to college in STEM despite scoring the same kinds of grades? Yes. Is there evidence that some of these problems (anxiety about maths, for example) can start with perceptions of teachers in primary school? Yes. Is there a problem?
And the question is always, if your previous exposure has not been fair, then is it reasonable to pick an arbitrary level of course that would be fair to people who haven’t been discriminated against? For years, racism was justified by culturally-based testing that could not be performed at the same level by people outside the culture – which was then used to restrict their access to the culture.
To me, that statement about exposure summarises everything that is wrong with glib arguments about constructing equal opportunity. If we’re going for a big job and there’s a corporate ‘interview dinner’ for 20 people, then we’ll all be on our best behaviour at dinner. For someone to lose the job because nobody showed them how to use a finger bowl or because their family uses a knife in the ‘other’ way, is to provide an equal exposure in the present that is blatantly unfair because it doesn’t take into account the redress of previous bias to bring people up to the point where it is really equal opportunity.
I think history supports me in the statement that we have been proved wrong every other time we’ve tried to segregate human ability and talent based on fixed physical abilities that were assigned at birth. Isn’t it about time we started investing all of our effort into producing truly equal opportunity for everyone?
Great post over on Mark Guzdial’s blog on the work being done by Barbara Ericson on Computing Summer Camps. You should head on over and read it (not you, Mark, but thank you!) but the core message is so useful and transferable that I wanted to reiterate it here. Student activities that foster engagement, participation and skill development are very popular but, to be successful, you have to make sure that you do them right. I had a chance to see Barb present when she and Mark were in Adelaide and her talk was really helpful because it was informative but also really, really useful. Too many times I’ve seen people talk a great theory at me but without giving me any starting points. Her talk, and the paper, highlight good practices with a strong basis. Here are the three points that capture why Barb’s summer camps are so good – with my own commentary added somewhat superfluously.
- Effectiveness is essential. Measuring student performance is vital to showing that students do improve – in attitudes and knowledge. If the camp isn’t effective in either increasing engagement or driving knowledge, then why are we bothering? I’m, going to mention MIKE again here – Measurement Is the Key to Everything. If we don’t measure, we have no idea what has succeeded or how we can make it work.
- The program is sustainable and will keep going after the first flush of money runs out. This is an enormous problem with so many of the programs I’ve seen – they work beautifully while the big cash is available and disappear when it dries up. Barb’s Summer Camps are sustainable as a whole because she’s done this for long enough to get some great rules of thumb for keeping enough money in from key groups to allow an investment in slightly smaller groups (such as using a large residential middle school camp to offset the costs of a smaller high school camp).
- The camps can be run by other people and still be successful. This replicability is another thing that’s frequently missing from our courses. All of my materials should be able to survive me moving on but, too often, they come close but I don’t quite capture all of the details – although I strive to. Barb’s aim is to have these programs running lots of places and, by making the material available and providing seed grants, there are now 11 more camps around Georgia, returning similar results in terms of success. There is only one Barb, but at the moment we have a 12-fold increase in ‘BarbCamposity’ through scaling. If Computer Scientists should be good at anything, it’s leveraging amplifiers to allow us to be in more than one place at once.
There are so many other places we can apply these principles and, most importantly, it identifies the focus of our efforts as educators – I don’t want my students to need me all the time, I want to bring information to them that is sound, that extends them and that supports them for years to come. By making sure that my material is effective, identifying needs, measuring impacts, I avoid wasting my time. By developing sustainable programs, which aren’t resource heavy, I can keep going whether we’re getting big dollars, small dollars, no dollars or (shudder) negative dollars as we slash budgets to ride out troubled times. Finally, my making my course so self-contained and good that someone else can teach it and someone else WANTS to teach it, I can go on to the next thing I want to do. This is liberating – I’m not writing myself out of a job, I’m giving myself the scope to pursue new techniques, to share my knowledge (such as it is) with other educators and to spend my time where it’s most needed.
Please go and read the blog post, and the paper, because both are really good and I’m only shadowing them here. One of the things I love about the vitality of the CSE community is that I can interact with, and learn so much from, people like Mark and Barb, but also share it with you – efficiently, sustainably and (given that I’m reflagging) in a replicable manner.