MIT Media Laboratory

ch@media.mit.edu

Presented at the symposium "Varieties of Programming Experience," AERA 2001, Seattle.

(Figures may be found at the end of this document)

Flogo is a new programming language designed to facilitate children’s programming of robot behaviors. Flogo’s "live" interface and visual dataflow representation are intended to make programs more understandable, accessible and modifiable. This presentation reports three themes that have surfaced in exploratory research:

1. Temporal structure emerges as an important conceptual domain;
2. Real-time visual feedback introduces a new kind of learning about programs, which might be called "Gibsonian" learning;
3. Episodes of successful tinkering prompt reflection on the nature and value of tinkering-friendly programming languages.

This symposium takes the construction of process to be a fundamental attribute of programming. But what forms can this take? Different kinds of programming languages offer qualitatively different tools and representations for constructing and understanding process. Different program genres and domains of application demand different sorts of properties in the processes that are constructed.

Robotics is a relatively new domain for programming by children, but a domain that is quickly gaining interest among educators, researchers, and especially children. In general, children love to build robots and create behaviors for them. In doing so, they can learn on many levels – about science, engineering, design, and artistic expression, as well as programming (Druin, 2000; Resnick, Berg, & Eisenberg, 2000). In this rich mixture, however, programming may not always figure prominently. Children may focus their efforts on imaginative or elaborate physical forms, or on mechanical engineering problems. Robot behaviors may be pre-packaged, or provided by an expert mentor; or, the child may construct extremely simple behaviors. Programming a robot to behave in interesting or intelligent ways – in ways that have some apparent autonomy – can be surprisingly hard. Like generations of AI researchers before them, children often find that goals must be repeatedly scaled back as the complexity of seemingly simple behaviors is revealed. The Flogo project attempts to empower children to construct more sophisticated behaviors than they have so far been able to do.

One might ask: If programming is already hard for children, why draw them into a kind of programming that may be even harder? Leaving aside the question of how one would judge whether it really is harder, here are four reasons. First, some of the difficulties of robotics programming will abate as we evolve languages better suited to the task. Second, some of these language improvements may benefit conventional programming as well, shedding new light on old problems. Third, as some of the difficulties of constructing behavior come into better focus, we may find that they lead to profound ideas that deserve a more central role in future education. Finally, children’s delight in bringing their constructions to life brings a fresh and optimistic energy to the programming enterprise.

Robotics for children has been a focus of work at the Media Lab for many years. This work has produced a lineage of robotics control hardware, of which the current representative is the "cricket," a control computer not much bigger than a nine-volt battery, with an architecture designed for easy expansion and adaptation (Martin, Mikhak, et al., 2000). Alongside the hardware development there have been explorations of a variety of programming approaches for robotics, ranging from straightforward adaptations of the Logo language, to multitasking extensions of Logo (Resnick, 1990), iconic program-construction tools (Begel, 1996), and programming by physically wiring up simple logic components (Hogg, Martin, & Resnick, 1991). Combining ideas from these and other programming languages, the Flogo project attempts to make robotics programming more accessible and rewarding, so that learners can construct more sophisticated robot behaviors, and learn more by doing so.

The long-term vision for Flogo is of a "low threshold, high ceiling" environment making optimal use of visual and textual representations to provide a smooth ramp for progress from novice to advanced levels of robotics programming.

As the name suggests, Flogo supports the representation of process in terms of data flow. Flogo is also "visual" (more precisely, diagrammatic), in the sense that it supports a box-and-wires notation similar to that of many visual dataflow languages. The research reported in this paper focuses on Flogo’s visual representations. However, it is important to note that Flogo also integrates textual and procedural process representations within the same environment, and future development and research will emphasize this. Visual and dataflow programming paradigms are known to have weaknesses of their own in comparison to textual and procedural ones (see e.g. Green & Petre, 1992; Whitley, 2000). Even scaled-back conjectures that their representations are superior for certain types of tasks have not been easy to confirm (Good, 1996). Flogo’s primary commitment is to meeting the needs of learners in the context of constructing behaviors, and its representations are evolving as those needs become clearer. Dataflow representations are used because they express some kinds of real-time processing naturally and intuitively, and because they present particular opportunities for "liveness" in the interface (see below). Our expectation is that text and diagrams, and procedural and dataflow process specifications, will serve best when they can be freely combined, making the most appropriate tools available for each part of a programming task, and for each learner. In this sense, the project’s stance might be described, with tongue somewhat in cheek, as "post-paradigmatic."

The characteristic of Flogo on which our hopes are most pinned is "liveness" — meaning that instead of existing as an inert object that is first edited and later run, a Flogo program can be active, and its activity visible, even as you are building or modifying it. Like a spreadsheet, the Flogo environment updates immediately to any change of inputs or relationships; however, the situation is more complex, because a program – especially a program for controlling behavior in time – changes over time in ways that a spreadsheet does not. When robotics programming is done in a live programming environment, programmer, robot (cum environment) and program enter into a dance in which all participants are simultaneously and mutually responsive. Some very basic consequences of this are described in this paper.

Dataflow, visual representations, and liveness go well together, but they are logically independent. Several existing children’s robotics languages, such as LogoBlocks and RoboLab, are visual or diagrammatic – but none is truly a dataflow language. Boxer is a comparatively live environment for textual, procedural programming. The three characteristics do co-occur in some other languages, such as LabView (National Instruments) and the robot-programming game MindRover (Cognitoy). All three characteristics are themselves complex, inviting subtle but important distinctions — but this is not the place for them.

One additional property is essential to the Flogo vision. Flogo is an extensible language, supporting the construction of re-usable, recombinable, modifiable, higher-level tools, complete with their own user interfaces. In this respect Flogo follows in the footsteps of Boxer and the "open toolsets" philosophy (diSessa, 1997).

Fig. 1 shows a simple robotic vehicle and a Flogo program that enables the vehicle to drive safely on a tabletop. The vehicle has downward-pointing reflectance sensors at its front left and right corners. These sensors can be used to tell when the corner has run off the edge of the table. In the Flogo program, the sensors are represented by the small components at the far left. From each component a wire carries the current sensor value to a comparison operator, which compares the sensor reading to a cutoff value. The output of the comparison will be 1 when that sensor is on the table and 0 when it is off the table. This output is fed into the on/off control of the motor on the robot’s opposite side. The net result is that the vehicle will avoid falling off the table. If it approaches the table edge at an angle, only one wheel will be inhibited, and the robot will turn away from the precipice. If the robot approaches the table edge on the perpendicular, then both sensors will go off the table and the vehicle will simply stop. This weakness is improved in the program in Fig. 2, which makes the robot back up and turn when both sensors are off the table.

Another important entity in Flogo is the "pulse," which also travels on wires and can be used to signal events and initiate actions. Fig. 3 shows two simple Flogo pulse-based idioms or "plans" that were used repeatedly by students in a variety of situations.

Fig. 4 illustrates one way to extend the language: a useful construction can be adapted and encapsulated within a component, which can be opened or closed.

Work to date has concentrated near the novice end of the spectrum of target users. The language has been tested with sixteen 10- and 11-year-olds with varied previous experience of Logo Microworlds programming. Eight children participated in a two-day "robotic arts" workshop. Eight more, plus one of the first group, participated in a four-day (20 hour) workshop which was preceded and followed by informal clinical sessions in which children, in pairs, were introduced to new language features and took on a variety of programming and problem-solving challenges.

In conjunction with Flogo, we use "cricket" control computers, which are the most recent in a long lineage of robotics control devices developed at the Media Lab (Martin, Mikhak, et al., 2000). Children use Flogo to control cricket computers through a constant communications link to a desktop computer, so that the operation of the program can be seen and adjusted in real time. The physical constructions are built mostly in Lego, with some additional craft materials. We would like to offer a "way in" to robotics for as many children as possible, so we try to support a variety of creative agendas. In addition to the more typical humanoids and vehicles, children have built such things as a model fairground ride, a "gardener’s helper" which spreads seeds by spinning, and assorted kinetic sculptures with interesting movement and light patterns. In the second workshop, all the children collaborated to construct mechanized versions of scenes from a storybook (Seeger & Hays, 1994): These were used to make a video of the story.

Children have mostly used Flogo to build programs of their own. In addition, I occasionally presented them with small puzzles, both to probe their understanding and to offer another way to learn about the language. In a series of "Indiana Jones" puzzles, the children are presented with a complete Flogo program that controls a Lego treasure chest (Fig. 5 shows one of these puzzles). The challenge is to manipulate the two sensors in a way that opens the secret compartment. Often children can get the compartment open in a few seconds, simply by aggressive clicking and poking. Then I explain the full challenge: find a way to open the compartment without letting the beeper go off once (the beeper means Indiana Jones has been bitten by a snake), and whenever you’re ready, you can play just once, "for keeps." These rules are readily agreed to and prompt a more careful investigation of the puzzle.

Flogo makes a basic distinction between two kinds of signals, corresponding to two structurally different ways that information can be embedded in time. A data signal is a steady stream of information representing the variation of some quantity over time. A pulse is an instantaneous signal carrying no informational content other than the time at which it occurs. Pulses are used to report events and to initiate actions.

Children were not always clear on the distinction between pulse and data. A common context for the problem is in situations where the program detects a sensor crossing a threshold value. It is easy to confuse the event of the sensor value crossing over the threshold with the span of time during which the sensor stays over the threshold value.

So, for example, when students confronted the mystery program in Fig. 4., they would expect the "Which" component to remain in the downward position as long as the sensor was pressed. This would happen while students were actually playing with the program, and manipulating the sensors. They could observe a relationship between the switch position and the sensor’s value going under the threshold. They would say things like "the switch needs the sensor to be below 100," and make corresponding action errors, holding down the sensor value well in advance, instead of timing the press. "Backsliding" was observed: when asked to look closely, a subject might identify that the pulse happens when the sensor value crosses over, but then repeat earlier errors. Also observed was correct action together with incorrect description: children saying that the switch is open while the sensor value is below 100, but in fact timing their movements to make the value cross under at just the right time.

Related errors occurred in other contexts:

There are several layers to sort out in this phenomenon. Of course, part of it is kids’ understanding of the pulse/data distinction in Flogo’s ontology. But it also appears that kids are not practiced in distinguishing events and durations when analyzing real-world processes.

The distinction between events and durations is an important part of understanding how to use multiprocess programming languages. In a study of children’s understanding of MultiLogo, a text-based concurrent programming language, Resnick (1990) observed a variety of "synchronization bugs," in which children either expected that one agent, having asked another agent to do an action, would wait until the action was finished (whereas in fact it would start immediately), or, conversely, that an agent could start a time-consuming action of its own and then ask another agent to do something in parallel (whereas in fact the second agent doesn’t get asked until the action is finished). Again, the difficulty is in the general area of sorting out instantaneous versus time-consuming actions, as they are coordinated among different processes. Children also had trouble with agents executing the "onfor" command. They thought that halting the agent would also halt the motor (whereas in fact, halting the agent before it could turn off the motor led to the motor staying on indefinitely). Here, children fail to realize that an apparently continuous engagement of two processes is actually nothing more than two instantaneous interactions (Fig. 6).

As described so far, the event-duration and pulse-data distinctions, and their cousins the synchronization bugs, look like issues specific to programming in parallel or multiprocess languages. And they are indeed more visible there. But similar phenomena do occur in other contexts. Fred Martin (personal communication) has observed many difficulties with what he calls the "edge/level distinction," among schoolchildren and college students, programming robots in a variety of procedural languages. The fact that the program has only one process does not remove the difficulty, because in real-time programming the computer’s processes (one or more) must be coordinated with all the events and processes in the world that the robot or device is meant to interact with. To do this kind of programming in a single-process language, you have to use sophisticated techniques, such as polling and explicit state representations, to avoid getting your single process trapped in one task to the exclusion of others. So a multiprocess language may in fact serve novices better by taking the bull by the horns.

We can also note the relevance of a well-known misconception of novice programmers, that a while loop will exit anytime its looping condition becomes false, whereas the condition is actually checked just once per iteration. In other words, does the condition apply over the duration of the loop, or by means of discrete condition-checking events. So even when there really is only one process, the naïve expectation seems to imply multiple processes, so process coordination is still an issue!

Widening the circle even farther, I think all of this can be seen as a discrete analog to issues that students face in learning about the mathematics of change. Pulse relates to data somewhat the way velocity, say, relates to distance. In each instance we have one kind of information which is associated with changes in other information. The comparison is complex, and I can’t yet offer a direct practical connection. But the generic similarity may lend a bit more credence to the suggestion I want to make, which is that there is a set of fundamental ideas here – we might label them "temporal structure" – that merit children’s attention and ours.

DiSessa (1986) has described how programming competence can be based on a patchwork of partial understandings. The understandings include structural knowledge (knowing something about the mechanics of computation, and the ability to predict, on this basis, how some computations will run) and functional knowledge (context-based "how to" knowledge for achieving desired ends), as well as schemas and expectations carried over from other domains. Development as a programmer will entail expansion, interconnection and reconciliation of these partial understandings, including both structural and functional understandings.

This variety of types of knowledge is apparent in children’s use of Flogo. Here’s an example of functional knowledge. All the children know by now that if you want a button to turn a motor on indefinitely, instead of just as long as the button is pressed, then you need to put a switch between them. This knowledge works for kids – they can use it to build working programs – even though most of them are hazy on what kind input the switch takes (pulses that say when to turn on and off) and what kind of output it generates (data about the current on/off status).

Now, what kinds of knowledge, and what ways of gaining knowledge, are we adding to the mix when children have a completely new way to interact with programs? Consider the Indiana Jones puzzles mentioned earlier. As I said, the challenge of playing for stakes, which requires the ability to open the compartment reliably, with no missteps, encouraged careful investigation by the children. However, it would be a mistake to call this investigation a mechanical analysis of the puzzle. There is some mechanical thinking (varying from learner to learner), but at least as important is a process of gradually identifying useful visual cues, and working how to coordinate one’s actions with these cues. I think of this as "Gibsonian" learning, after the psychologist James J. Gibson, who investigated how visual cues can support adaptive action in the world without the use of mental representation (Gibson, 1966).

For example, in solving another Indiana Jones puzzle, one boy developed a mantra, "one...two...three… send it across" – a terse guide to what to do and what to pay attention to in the program. The slow counting marked out how long a particular touch sensor needed to be pressed; the "it" that was sent across was a pulse that needed to get to a target part of the program, past some logical obstacles which the "one…two…three" action would temporarily clear. By borrowing the mantra, his partner, who had made even fewer mechanical interpretations of the puzzle, began to be able to open the compartment as well.

What good is this kind of learning and knowledge? I think that through these interactions kids are

To clarify what I mean by #3, consider the big box, called a "back-forth", in the lower left of the Indiana Jones program (Fig. 5). One pulse into this box will make the compartment open; the next one will make it close, and so on. Most kids did not try to analyze how it worked, but they knew what it did, and that it was the target in these puzzles. Based only on this exposure, two pairs spontaneously used a back-forth in their own constructions: one to make a car drive back and forth; and the other to make two arms of a kinetic sculpture wave one way, then the other. (Note that these are different from the original application, so it appears that they had abstracted the mechanical essence of the back-forth.)

Figure 7 shows one of these adaptations. The pulse-repeat sends a pulse to the back-forth once every second (the 10 is in units of 1/10 second). Inside the back-forth, the first component has the job of ignoring any pulses that follow within 3-1/2 seconds of the previous pulse. (This prevents the movement from being interrupted in midcourse.) So only one pulse in four gets through. These details presumably escaped the notice of the programmer, who was happy with the overall behavior.

The fact that the programmer was able to get a successful program working without understanding everything about how it worked, i.e. by tinkering, is a good thing. Tinkering is valuable because it gives learners a chance to succeed, and to gain various kinds of knowledge and experience, while their structural knowledge and planning skills are still fragmentary. Structural knowledge gains not only time to develop, but also a framework of other kinds of knowledge with which it can be "webbed" (Noss & Hoyles, 1997). Programming will become more learnable as we make its deep ideas more accessible, but also as we allow children more time and context to learn them, by supporting tinkering.

If we take support for tinkering to be desirable in a programming language, what are some of the characteristics that would contribute to this support? One of them would surely be the ability to test programs, and parts of programs, in the "live" way that kids investigated the Indiana Jones puzzles. "Gibsonian learning" about these components would be one way for the tinkerer to sort out what aspects of execution to pay attention to, and how to adjust for them.

Flogo has a second language property that appears to support tinkering: functional and non-functional parts of a program can coexist in the same program, without interfering with each other. This property can greatly reduce the price of experimentation. It is easy to add new components and wires to a program, without fear that they will introduce damaging side effects. Fig. 8 shows a snapshot of an extended programming session. The functional core of the program is surrounded by partial constructions, some abandoned, others in process. The children worked iteratively, making a few modifications (some planful and some haphazard), testing them onscreen or via the robot, interpreting the results (in terms of structure, or just success), and modifying again. Because they never had to "break" the working parts of the program in order to try new ideas, forward and lateral progress far outweighed regress. By the end of the session, real progress had been made.

To put in another way, I am suggesting that Flogo’s "live" dataflow representations help to make programs less fragile. The principle could be summed up as follows: if your program behaves as you build it, and if each part behaves where you built it, then the program breaks less when you tinker with it.

This session also happened to be the boys’ first encounter with a new feature of Flogo. After seeing children’s difficulties with pulses and data, I changed Flogo to make the distinction more explicit. Pulse and data nodes look different, as do the wires between them. Automatic, convenient conversion of levels to edges was abandoned (the transition from zero to nonzero had been interpreted as a pulse) and Flogo now refuses to connect a data node to a pulse node. I thought that this might force the issue of pulse vs. data and get kids thinking and talking more about the issue. At the beginning of the session, I presented and explained the new system. Without further discussion of the nature of pulse and data the boys proceeded confidently on their project. Finding, as time went by, that some attempted wirings were now prevented, one of the boys good-naturedly remarked, "these orange nodes are really messing us up… they make it harder." In fact, though, only dead ends were thwarted, and I believe they got their constructions working more quickly than they would have with the earlier version. A language change developed with structural understanding in mind thus turned to have a different kind of value: with no immediate effect on structural understanding, it nevertheless helped to make the tinkerers’ search for useful behaviors more efficient.

Of course, tinkering is an important phenomenon for many learners in many programming languages (Turkle & Papert, 1992). It is too early to judge Flogo’s tinkering-friendliness, not only for lack of data, but also because the language is still in its infancy. What is clearer at this point is the value of the design goal of a tinkering-friendly programming language, and the potential for visual dataflow representations to support tinkering. Flogo’s ultimate success in this department will depend on the potential of the paradigms it integrates, but also on many design decisions, small and large.

If programming is to fulfill its educational promise, we must continue to improve our understanding of programming is, and what it might be. The comparative approach is essential to the pursuit of this understanding. Programming experience comprises so many interacting variables: the computational paradigm, the way the paradigm is represented, the construction genres, tasks and challenges, and of course social context. Varying these, you can force an issue forward or gloss it over for a while. You can trade one problem off for another one. You can recruit different kinds of knowledge and intuition that kids may bring with them, and appeal to different interests. Through this variation one hopes to gain a sense of what ideas are most fundamental and empowering, and where they make the strongest contact with children.

As a visual, parallel, real-time programming language for robotics, Flogo brings forward the time dimension in programming. In the realm of structural understanding, issues of temporal coordination move front and center. In the realm of learning and program development, Flogo changes how our interactions with programs are organized in time. This expands the kinds of knowledge that children can bring to bear, and offers some promise that by enhancing the value of tinkering, we can providing a more nurturing environment for the growth of programming knowledge.

Thanks to Mitchel Resnick for guidance and support, and to Bakhtiar Mikhak for early collaborations and conversations that helped me find my way to this work. Thanks to Sam Davies and Hana Kim for contributions to the Flogo software effort. Many ideas in this paper and in Flogo have benefited from conversations with past and present members of the Media Lab’s Lifelong Kindergarten group, including Andy Begel, Fred Martin, Bakhtiar Mikhak, Mitchel Resnick, and Brian Silverman. Thanks to the staff, students, and parents of Milton Academy for their energetic participation in two robotics workshops and related interviews, to Cynthia Solomon for her collaboration in arranging and leading the workshops and making sense of the experience, and to Eric Solfisburg for video assistance.

Begel, A. (1996). LogoBlocks: A Graphical Programming Language for Interacting with the World. Advanced Undergraduate Project. Massachuesetts Institute of Technology. (http://www.cs.berkeley.edu/~abegel/mit/begel-aup.pdf)

diSessa, A.A. (1986). Models of computation. In D.A.Norman & S.W.Draper (Eds.) User-centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Erlbaum.

diSessa, A. A. (1997). Open toolsets: New ends and new means in learning mathematics and science with computers. Paper presented at the 21st Conference of the International Group for the Psychology of Mathematics Education, Lahti, Finland.

Druin, A., Ed. (2000). Robots for kids: Exploring new technologies for learning experiences. San Francisco: Morgan Kaufman / Academic Press.

Gibson, J.J. (1966). The senses considered as perceptual systems. London: George Allen & Unwin.

Good, J. (1996) The ‘right’ tool for the task: An investigation of external representations, program abstractions and task requirements. In W.D.Gray & D.A.Boehm-Davis (Eds.) Empirical studies of programmers: Sixth workshop. Norwood, NJ: Ablex.

Green, T. R. G., & Petre, M. (1992). When visual programs are harder to read than textual programs. Proc. Sixth European Conference on Cognitive Ergonomics (ECCE 6), pp. 167-180.

Hogg, D.W., Martin, F., & Resnick, M. (1991). Braitenberg creatures. E&L Memo No. 13. Epistemology and Learning Group, MIT Media Laboratory.

Martin, F., Mikhak, B., Resnick, M., Silverman, B., & Berg, R. (2000). To Mindstorms and beyond: Evolution of a construction kit for magical machines. In A. Druin (Ed.) Robots for kids: Exploring new technologies for learning experiences. San Francisco: Morgan Kaufman / Academic Press.

Noss, R., & Hoyles, C. (1996). Windows on mathematical meanings: Learning cultures and computers. Dordrecht, the Netherlands: Kluwer Academic Publishers.

Portsmore, M. (1999). Robolab: Intuitive robotic programming software to support lifelong learning. Apple Learning Technology Review, Spring-Summer 1999.

Resnick, M. (1990). MultiLogo: A study of children and concurrent programming. Interactive Learning Environments, 1(3), 153-170.

Resnick, M., Berg, R., & Eisenberg, M. (2000). Beyond black boxes: Bringing transparency and aesthetics back to scientific investigation. Journal of the Learning Sciences, 9(1) pp.7-30.

Seeger, P., & Hays, M. (1994) Abiyoyo : based on a South African lullaby and folk story. (Reprint edition). New York : Aladdin Books.

Turkle, S., & Papert, S. (1992). Epistemological Pluralism and the Revaluation of the Concrete. Journal of Mathematical Behavior, Vol. 11, No.1, pp. 3-33.

Whitley, K.N. (2000). Empirical research of visual programming languages: An experiment testing the comprehensibility of LabView. Unpublished doctoral dissertation. Computer Science dept., Vanderbilt University.

Figure 1. A "Table Explorer" robot and a simple Flogo program to control it. When a sensor moves off the table, the diagonally opposite motor is inhibited.

Figure 2. A more complex Table Explorer program. If both sensors go off the table, one wheel goes backwards. (The second input node on the TIMOTORB component controls motor direction).

Figure 3. Early Flogo idioms or "Plans," using a Pulse-delay (A) to turn something off automatically, and (B) to emit repeated pulses. Red dot shows progress of pulse through delay. Wires flash red momentarily when a pulse passes through.

Figure 4. Encapsulation within a Pulse-Repeat component (shown opened and closed).

Figure 5. "Indiana Jones" puzzle. The challenge is to trigger the motor and open a secret compartment. (This is the second puzzle in a series of three.)

Figure 6. Tricky distinctions in temporal coordination, illustrated as timeline diagrams.

Figure 7. Example of programming by tinkering. Successfully re-using a Back-Forth in a non-standard way (three out of four input pulses will be ignored).

Figure 8. Another tinkering example. Work-in-progress shows non-functional , experimental and test code coexisting with functional core.