Authenticity and Challenge: Software Engineering Projects Where Failure is an Option

It’s nearly the end of semester and that means that a lot of projects are coming to fruition – or, in a few cases, are still on fire as people run around desperately trying to put them out. I wrote a while about seeing Fred Brooks at a conference (SIGCSE) and his keynote on building student projects that work. The first four of his eleven basic guidelines were:

  1. Have real projects for real clients.
  2. Groups of 3-5.
  3. Have lots of project choices
  4. Groups must be allowed to fail.

We’ve done this for some time in our fourth year Software Engineering option but, as part of a “Dammit, we’re Computer Science, people should be coming to ask about getting CS projects done” initiative, we’ve now changed our third year SE Group Project offering from a parallel version of an existing project to real projects for real clients, although I must confess that I have acted as a proxy in some of them. However, the client need is real, the brief is real, there are a lot of projects on the go and the projects are so large and complex that:

  1. Failure is an option.
  2. Groups have to work out which part they will be able to achieve in the 12 weeks that they have.

For the most part, this approach has been a resounding success. The groups have developed their team maturity faster, they have delivered useful and evolving prototypes, they have started to develop entire tool suites and solve quite complex side problems because they’ve run across areas that no-one else is working in and, most of all, the pride that they are taking in their work is evident. We have lit the blue touch paper and some of these students are skyrocketing upwards. However, let me not lose sight of one our biggest objectives, that we be confident that these students will be able to work with clients. In the vast majority of cases, I am very happy to say that I am confident that these students can make a useful, practical and informed contribution to a software engineering project – and they still have another year of projects and development to go.

The freedom that comes with being open with a client about the possibility of failure cannot be overvalued. This gives both you and the client a clear understanding of what is involved- we do not need to shield the students, nor does the client have to worry about how their satisfaction with software will influence things. We scaffold carefully but we have to allow for the full range of outcomes. We, of course, expect the vast majority of projects to succeed but this experience will not be authentic unless we start to pull away the scaffolding over time and see how the students stand by themselves. We are not, by any stretch, leaving these students in the wilderness. I’m fulfilling several roles here: proxying for some clients, sharing systems knowledge, giving advice, mentoring and, every so often, giving a well-needed hairy eyeball to a bad idea or practice. There is also the main project manager and supervisor who is working a very busy week to keep track of all of these groups and provide all of what I am and much, much more. But, despite this, sometimes we just have to leave the students to themselves and it will, almost always, dawn on them that problem solving requires them to solve the problem.

I’m really pleased to see this actually working because it started as a brainstorm of my “Why aren’t we being asked to get involved in more local software projects” question and bouncing it off the main project supervisor, who was desperate for more authentic and diverse software projects. Here is a distillation of our experience so far:

  1. The students are taking more ownership of the projects.
  2. The students are producing a lot of high quality work, using aggressive prototyping and regular consultation, staged across the whole development time.
  3. The students are responsive and open to criticism.
  4. The students have a better understanding of Software Engineering as a discipline and a practice.
  5. The students are proud of what they have achieved.

None of this should come as much of a surprise but, in a 25,000+ person University, there are a lot of little software projects on the 3-person team 12 month scale, which are perfect for two half-year project slots because students have to design for the whole and then decide which parts to implement. We hope to give these projects back to them (or similar groups) for further development in the future because that is the way of many, many software engineers: the completion, extension and refactoring of other people’s codebases. (Something most students don’t realise is that it only takes a very short time for a codebase you knew like the back of your hand to resemble the product of alien invaders.)

I am quietly confident, and hopeful, that this bodes well for our Software Engineers and that we still start to seem them all closely bunched towards the high achieving side of the spectrum in terms of their ability to practice. We’re planning to keep running this in the future because the early results have been so promising. I suppose the only problem now is that I have to go and find a huge number of new projects for people to start on for 2013.

As problems go, I can certainly live with that one!

Fred Brooks: Building Student Projects That Work For Us, For Them and For Their Clients

In the Thursday keynote, Professor Brooks discussed a couple of project approaches that he thought were useful, based on his extensive experience. Once again, if you’re not in ICT, don’t switch off – I’m sure that there’s something that you can use if you want to put projects into your courses. Long-term group projects are very challenging and, while you find them everywhere, that’s no guarantee that they’re being managed properly. I’ll introduce Professor Brooks points and then finish with a brief discussion on the use of projects in the curriculum.

Firstly, Brooks described a course that you might find in a computer architecture course. The key aspect of this project is that modern computer architecture is very complex and, in many ways, fully realistic general purpose machines are well beyond the scope of time and ability that we have until very late undergraduate study. Brooks works with a special-purpose unit but this drives other requirements, as we’ll see. Fred’s guidelines for this kind of project are:

  1. Have milestones with early delivery.

    Application Description
    Students must provide a detailed application description which is the most precise statement that the students can manage. A precise guess is considered better here than a vague fact, because the explicit assumption can be questioned and corrected, where handwaving may leave holes in the specification that will need to be fixed later. Students should be aware of how sensitive the application is to each assumption – this allows people to invest effort accordingly. The special-purpose nature of the architecture that they’re constructing means that the application description has to be very, very accurate or you risk building the wrong machine.

    Programming Manual (End of the first month as a draft)
    Another early deliverable is a programming manual – for a piece of software that hasn’t been written yet. Students are encouraged to put something specific down because, even if it’s wrong, it encourages thought and an early precision of thought.

  2. Then the manual is intensely critiqued – students get the chance to re-do it.
  3. The actual project is then handed in well before the final days of semester.
  4. Once again the complete project goes through a very intense critique.
  5. Students get the chance to incorporate the changes from the critique. Students will pay attention to the critique because it is criticism on a live document where they can act to improve their performance.

The next project described is a classic Software Engineering project – team-based software production using strong SE principles. This is a typical project found in CS at the University level but is time-intensive to manage and easy to get wrong. Fred shared his ideas on how it could be done well, based on running it over 22 years since 1966. Here are his thoughts:

  1. You should have real projects for real clients.

    Advertise on the web for software that can be achieved by 3-5 people during a single semester, which would be useful BUT (and it’s an important BUT) you can live without. There must be the possibility that the students can fail, which means that clients have to be ready to get nothing, after having invested meeting time throughout the project.

  2. Teams should be 3-5, ideally 4.

    With 3 people you tend to get two against one. With five, things can get too diffuse in terms of role allocation. Four is, according to Fred, just right.

  3. There should be lots of project choices, to allow teams choice and they can provide a selection of those that they want.
  4.  Teams must be allowed to fail.

    Not every team will fail, or needs to fail, but students need to know that it’s possible.

  5. Roles should be separated.

    Get clear role separations and stick to them. One will look after the schedule, using the pitchfork of motivation, obtaining resources and handling management one level up. One will be chief designer of technical content. Other jobs will be split out but it should be considered a full-time job.

  6. Get the client requirements and get them right.

    The client generally doesn’t really know what they want. You need to talk to them and refine their requirements over time. Build a prototype because it allows someone to say “That’s not what I want!” Invest time early to get these requirements right!

  7. Meet the teams weekly.

    Weekly coaching is labour intensive and is mostly made up of listening, coaching and offering suggestions – but it takes time. Meeting each week makes something happen each week. When a student explains – they are going to have to think.

  8. Early deliverable to clients, with feedback.

    Deadlines make things happen.

  9.  Get something (anything) running early.

    The joy of getting anything running is a spur to further success. It boosts morale and increases effort. Whatever you build can be simple but it should have built-in stubs or points where the extension to complexity can easily occur . You don’t want to have to completely reengineer at every iteration!

  10. Make the students present publicly.

    Software Engineers have to be able to communicate what they are doing, what they have done and what they are planning to do – to their bosses, to their clients, to their staff. It’s a vital skill in the industry.

  11. Final grade is composed of a Team Grade (relatively easy to assess) AND an Individual Grade (harder)

    Don’t multiple one by the other! If effort has been expended, some sort of mark should result. The Team Grade can come from the documentation, the presentation and an assessment of functionality – time consuming but relatively easy. The Individual Grade has to be fair to everyone and either you or the group may give a false indication of a person’s value. Have an idea of how you think everyone is going and then compare that to the group’s impression – they’ll often be the same. Fred suggested giving everyone 10 points that they allocated to everyone ELSE in the group. In his experience, these tallies usually agreed with his impression – except on the rare occasion when he had been fooled by a “mighty fast talker”

This is a pretty demanding list. How do you do tasks for people at the risk of wasting their time for six months? If failure is possible, then failure is inevitable at some stage and it’s always going to hurt to some extent. A project is going to be a deep drilling exercise and, by its nature, cannot be the only thing that students do because they’ll miss out on essential breadth. But the above suggestions will help to make sure that, when the students do go drilling, they have a reasonable chance of striking oil.