Is an IDE an E3? Maybe an E2?

Earlier, I split the evaluation resources of a course into:

  • E1 (the lecturer and course designer),
  • E2 (human work that can be based on rubrics, including peer assessment and casual markers),
  • E3 (complicated automated evaluation mechanisms)
  • E4 (simple automated evaluation mechanisms, often for acceptance testing)

E1 and E2 everyone tends to understand, because the culture of Prof+TA is widespread, as is the concept of peer assessment. In a Computing Course, we can define E3 as complex marking scripts that perform amazing actions in response to input (or even carry out formal analysis if we’re being really keen), with E4 as simple file checks, program compilation and dumb scripts that jam in a set of data and see what comes out.

But let’s get back to my first year, first exposure, programming class. What I want is hands-on, concrete, active participation and constructive activity and lots of it. To support that, I want the best and most immediate feedback I can provide. Now I can try to fill a room with tutors, or do a lot of peer work, but there will come times when I want to provide some sort of automated feedback.

Given how inexperienced these students are, it could be a quite a lot to expect them to get their code together and then submit it to a separate evaluation system, then interpret the results. (Remember I noted earlier on how code tracing correlates with code ability.)

Thus, the best way to get that automated feedback is probably working with the student in place. And that brings us to the Integrated Development Environment (IDE). An IDE is an application that provides facilities to computer programmers and helps them to develop software. They can be very complicated and rich (Eclipse), simple (Processing) or aimed at pedagogical support (Scratch, BlueJ, Greenfoot et al) but they are usually made up of a place in which you can assemble code (typing or dragging) and a set of buttons or tools to make things happen. These are usually quite abstract for early programmers, built on notional machines rather than requiring a detailed knowledge of hardware.

Screen Shot 2016-01-31 at 4.12.01 PM

The Processing IDE. Type in one box. Hit play. Rectangle appears.

Even simple IDEs will tell you things that provide immediate feedback. We know how these environments can have positive reception, with some demonstrated benefits, although I recommend reading Sorva et al’s “A Review of Generic Program Visualization Systems for Introductory Programming Education” to see the open research questions. In particular, people employing IDEs in teaching often worry about the time to teach the environment (as well as the language), software visualisations, concern about time on task, lack of integration and the often short lifespan of many of the simpler IDEs that are focused on pedagogical outcomes. Even for well-established systems such as BlueJ, there’s always concern over whether the investment of time in learning it is going to pay off.

In academia, time is our currency.

But let me make an aesthetic argument for IDEs, based on the feedback that I’ve already put into my beautiful model. We want to maximise feedback in a useful way for early programmers. Early programmers are still learning the language, still learning how to spell words, how to punctuate, and are building up to a grammatical understanding. An IDE can provide immediate feedback as to what the computer ‘thinks’ is going on with the program and this can help the junior programmer make immediate changes. (Some IDEs have graphical representations for object systems but we won’t discuss these any further here as the time to introduce objects is a subject of debate.)

Now there’s a lot of discussion over the readability of computer error messages but let me show you an example. What’s gone wrong in this program?


Screen Shot 2016-01-31 at 4.20.22 PM

See where that little red line is, just on the end of the first line? Down the bottom there’s a message that says “missing a semicolon”. In the Processing language, almost all lines end with a “;” so that section of code should read:


Did you get that? That missing semicolon problem has been an issue for years because many systems report the semicolon missing on the next line, due to the way that compilers work. Here, Processing is clearly saying: Oi! Put a semi-colon on the red squiggle.

I’m an old programmer, who currently programs in Java, C++ and Processing, so typing “;” at the end of a line is second nature to me. But it’s an easy mistake for a new programmer to make because, between all of the ( and the ) and the , and the numbers and the size and the rect… what do I do with the “;”?

The Processing IDE is functioning in at least an E4 mode: simple acceptance testing that won’t let anything happen until you fix that particular problem. It’s even giving you feedback as to what’s wrong. Now this isn’t to say that it’s great but it’s certainly better than a student sitting there with her hand up for 20 minutes waiting for a tutor to have the time to come over and say “Oh, you’re missing a semicolon.”

We don’t want shotgun coding, where random fixes and bashed-in attempts are made desperately to solve a problem. We want students to get used to getting feedback on how they’re going and using this to improve what they do.

Because of Processing’s highly visual mode, I think it’s closer to E3 (complex scripting) in many ways because it can tell you if it doesn’t understand what you’re trying to do at all. Beyond just not doing something, it can clearly tell you what’s wrong.

But what if it works and then the student puts something up on the screen, a graphic of some sort and it’s not quite right? Then the student has started to become their own E2, evaluating what has happened in response to the code and using human insight to address the shortfall and make changes. Not as an expert but, with support and encouragement, a developing expertise.

Feedback is good. Immediacy is good. Student involvement is good. Code tracing is linked to coding ability. A well-designed IDE can be simple and engage the student to an extent that is potentially as high as E2, although it won’t be as rich, without using any other human evaluation resources. Even if there is no other benefit, the aesthetic argument is giving us a very strong nudge to adopt an appropriate IDE.

Maybe it’s time to hang up the command line and live in a world where IDEs can help us to get things done faster, support our students better and make our formal human evaluation resources go further.

What do you think?

2 Comments on “Is an IDE an E3? Maybe an E2?”

  1. Kathi Fisler says:

    Reading this as a call for IDE designers to better account for
    learning, I agree entirely. Many IDEs aren’t designed from an
    explicit pedagogic perspective.

    I doubt that a student will become their own E2 unless the IDE’s
    feedback mechanisms are designed _very_ carefully towards pedagogy. In
    2011, we did a study of how students responded to
    pedagogically-designed error messages (in the DrRacket IDE, which had
    been designed for pedagogy with particular attention to tailoring
    error messages to the concepts/notional machine that students had
    learned so far). With certain kinds of errors, students entered a
    flailing mode, just trying various characters searching for something
    that might make sense. I don’t know how much actual knowledge
    students were able to take from that flailing-style typing. In part,
    wouldn’t the IDE need to prompt students to step back and reflect on
    what was wrong and what worked, if we actually expect students to
    absorb applicable knowledge from it?

    But reading this piece made me think of a different concern I have
    about the role of automated tools in grading. Automated grading works
    fine when students produce code that compiles and mostly works. But
    how serious is the impact on those who fail one of those criteria? A
    student who can’t get code to compile might actually understand
    something, but automated grading gives no credit for that. How much
    do we frustrate students who struggle with syntax by requiring them to
    get code through the E4 and E3 levels?

    I sometimes wonder if students shy away from CS classes in part
    because it is hard to call yourself “done” before the compiler
    agrees. When an assignment requires producing prose, a student can
    more or less stop when they think they’ve done enough. In CS (and
    other engineering disciplines), students can’t stop until the compiler
    thinks they’ve done enough. Should we be more supportive of students
    who can’t get to working code, but are getting something? Or is
    producing running code sufficiently important that we shouldn’t let
    students pass individual assignments without that?



    • nickfalkner says:

      Thanks for the comment. I agree that E2 activity may not be common but, in our case, because we were looking at graphic-focussed output, the human visual system and matching what the student wanted to what they got lurks (just) into the bottom of E2. Very specialised case but quite deliberately chosen to be so. 🙂

      Generally, flailing isn’t learning, as we know from the discussions of shotgun coding. It’s desperation, as you note.

      To your last point, I think that’s why the messages to get things working in Processing are so important. E4 without guidance to go forward is unnecessarily punitive and richer IDE feedback is an important part of that. The essential nature of good feedback is it tells you how to go further and E4 should help you to get to E3, not just tell you that you’ve not reached E4.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s