################################################ # # # ## ## ###### ####### ## ## ## ## ## # # ## ## ## ## ## ### ## ## ## ## # # ## ## ## ## #### ## ## ## ## # # ## ## ###### ###### ## ## ## ## ### # # ## ## ## ## ## #### ## ## ## # # ## ## ## ## ## ## ### ## ## ## # # ####### ###### ####### ## ## ## ## ## # # # ################################################ The following paper was originally presented at the Third Annual Tcl/Tk Workshop Toronto, Ontario, Canada, July 1995 sponsored by Unisys, Inc. and USENIX Association It was published by USENIX Association in the 1995 Tcl/Tk Workshop Proceedings. For more information about USENIX Association contact: 1. Phone: 510 528-8649 2. FAX: 510 548-5738 3. Email: office@usenix.org 4. WWW URL: https://www.usenix.org ^L ``Extending Tcl for Dynamic Object-Oriented Programming'' David Wetherall and Christopher J. Lindblad May 1995 This research was supported by the Advanced Research Projects Agency of the Department of Defense, monitored by the United States Air Force (AFSC, Rome Laboratory) under contract No. F30602-92-C-0019, and by the University of Western Australia under a Hackett Studentship. The authors can be reached at: MIT Laboratory for Computer Science, Room 505, 545 Technology Square, Cambridge, MA 02139; Tel: +1 617 253 7341; Email: djw@lcs.mit.edu. Telemedia Networks and Systems Group Laboratory for Computer Science Massachusetts Institute of Technology Abstract Object Tcl is an extension to the Tool Command Language (Tcl) for the management of complicated data types and dynamic object-oriented programming in general. We believe it is a worthy alternative to other object-oriented programming extensions (including [incr Tcl]) because it may be used dynamically, allows for per object specialization, has an economy of design and implementation, and provides a metaobject-based class system. Its design was driven by our VuSystem application needs to create a foundation with powerful abstraction and introspection capabilities, yet we sought to retain both the spirit and benefits of Tcl. This paper presents Object Tcl, emphasizing language design and implementation issues by comparing it with alternative systems. Keywords: object-oriented programming, Tcl, programming languages, [incr Tcl] 1 Introduction Object Tcl is an object-oriented extension to the Tool command Language (Tcl) that we created to meet the programming needs of our multimedia applications. A version embedded in the VuSystem has been in use for two years, and its success has prompted us to make a separate distribution. The many Tcl extensions for object-oriented programming (including [incr Tcl], CASTE and Objectify) attest to the popularity of the object approach. We present Object Tcl in light of them, as an alternative resulting from a different design philosophy, and as a summary of our input towards a minimum set of object extensions for Tcl. Table 1: Feature Comparison (omitted) Table 2: Usage Analogies (omitted) When designing our extension, we sought to extend Tcl with its own object-oriented programming facilities, as opposed to bringing an existing object language (such as C++, CLOS or Objective C) to Tcl. That is, we sought a small set of object primitives that would retain and build on Tcl's syntax, extensibility, simplicity, and compatibility with C. Each of these points distinguishes our work from [incr-Tcl], an extension that supplements Tcl with a structured programming environment modeled after C++. Our extension more closely resembles CASTE, though it uses Tcl-like syntax instead of CLOS-like syntax. In terms of other languages, our work has resulted in a compact, dynamic, and introspective language modeled after Flavors rather than C++. As with other extensions, we use a message-passing style to be consistent with Tk. However, unlike other extensions Object Tcl allows each object to be specialized, a capability seen in languages such as Self. This is an outgrowth of a strategy that we refer to as the object command approach. We believe it is well suited to Tcl/Tk since it directly supports the grouping of each widget object with its behaviors and related data. Our approach also shares goals with languages such as Dylan, which attempt to be practical on small machines while providing many of the language features found in CLOS and other advanced object systems. This paper presents Object Tcl, emphasizing the language design and implementation issues that we encountered. We begin with the object command approach for grouping data and operations in Tcl, and the basic objects that result from taking this approach to its conclusion. Then we consider the additional abstraction capabilities provided through inheritance mechanisms, and through a class system. This progression is intended to reflect increasing levels of both functionality and design, much as we followed in the development of Object Tcl. We assume a familiarity with object-oriented programming techniques and Tcl/Tk. 2 Comparison to Tcl/Tk & [incr Tcl] Before presenting Object Tcl, we provide a rough comparison with Tcl/Tk and [incr Tcl]. An analogy with Tcl/Tk helps to show how Object Tcl is designed to extend Tcl. An analogy with [incr Tcl] (the most popular of the object extensions) helps those familiar with it to gauge Object Tcl. Features Table 1 summarizes a comparison of features, and should be taken as a promise: by the end of the paper, each item in the table will be addressed. Many of the differences stem from the different biases of C++ and Flavors. [incr Tcl] classes are defined as a single block, while Object Tcl classes are defined incrementally across as many blocks as needed, typically one block per method and with few ordering constraints. This difference goes hand-in-hand with extensibility. Methods and classes are encouraged to be individually changed at any time in Object Tcl, while they tend to be grouped and declared once in [incr Tcl]. The support for run-time extensibility in conjunction with introspection is the main reason we describe Object Tcl as ``dynamic''. Tcl is designed so that much information on variables and procedures is accessible via Tcl commands and new variables and procedures can be created freely. The same is true of Object Tcl, further extended to allow attributes such as the class of an object to be discovered and changed as needed. The combination of introspection and extensibility permits a dynamic style of programming, where programs interrogate and modify programs. We believe this is a natural style for an object extension to Tcl, since it matches the capabilities of Tcl itself. Conversely, the static organization style of C/C++ programs is mainly a consequence of requiring that many decisions be made at compile-time, a consideration that is not appropriate for Tcl. Usage Table 2 summarizes a comparison of language syntaxes. Syntax often receives scant attention in evaluating a language, but we believe it is an important consideration here because we are attempting to extend an existing language, rather than to create a new one. Accordingly, Object Tcl mirrors Tcl command names where appropriate. A set method, for example, is used to create instance variables, rather than introduce a different notation. Its syntax is also designed to be compatible with the usage of Tk widgets. A second difference when compared to [incr Tcl] is that Object Tcl provides fewer ways of forming expressions, particularly for method invocation. This is because we attempted to include only a core set of facilities. For example, a special syntax for the automatic naming of objects (the #auto in [incr Tcl]) is not included, since it is easy to provide via a separate command. 3 Introducing Objects to Tcl The Tool Command Language (Tcl) is apt for combining primitive commands into complete applications because of its simplicity and extensibility. In Tcl, all commands and values are represented as strings. It is easy to pass data between the Tcl interpreter and commands written in C code in an application: the C code need only be able to convert the internal representations to and from strings. When data too complex to be readily converted to and from strings must be managed, the implementation may be pushed further into the command written in C. Using this approach, each object of complicated data is registered with the interpreter as a single command. Operations on an object are performed with subcommands by using the first argument to the command to specify the operation. We refer to this strategy as the object command approach, since each command may be interpreted as an object and each subcommand as a message to that object. It is implicit in Tk and the majority of object extensions, and has proved effective for structuring programs as well as for managing complicated data. Objects complement and extend the inbuilt Tcl organizations of lists, arrays, procedures, and commands. They provide a natural way to group related operations and data, and lead to object-oriented programming techniques. Object astack astack set things {} astack proc put {thing} { $self instvar things set things [concat [list $thing] $things] return $thing } astack proc get {} { $self instvar things set top [lindex $things 0] set things [lrange $things 1 end] return $top } astack put toast ==> toast astack get ==> toast astack destroy ==> {} Figure 1: Making a stack object by specializing a generic object. Objects Object Tcl formalizes the object command approach, allowing object models to be realized without replicating the infrastructure that would otherwise be necessary. Objects support the same flexibility in combining Tcl and C as does the Tcl language. Methods may be implemented in C or Tcl as appropriate, leading to objects with a mixture of implementation; however, only the Tcl interface is presented in this paper. CASTE also includes this capability, as will a forthcoming release of [incr Tcl]. Each object in Object Tcl is manipulated via a single command registered with the interpreter. Simple objects are created with the Object command, as shown in Figure 1. Here, the object astack is first created, then built up to implement a stack by adding one instance variable and two methods, and finally tested and destroyed. To understand the example, first notice that the astack object is individually crafted, without requiring a class definition for all stack objects. Not only may an object's data differ from every other object, its code may differ as well. This is in contrast to Tk widgets and other object extensions. Yet it is a natural extension of the object command approach (since objects are synonymous with commands and different commands typically hold different behaviours) and should lead to a natural style of Tcl programming. Table 3: Object Methods (omitted) Methods & Instance Variables The get and put methods of the astack object are introduced with the proc method, which mirrors the behavior of the Tcl proc command. When later invoked, methods appear as procedures with the addition of three special variables (Table 4) that describe the method context. For example, the reference to self in the method bodies holds the name of the object for which the method is executing. It allows other calls to the the same object; self is the equivalent of this in C++ and [incr Tcl]. Table 4: Method Environment (omitted) Similarly, the things instance variable is introduced with the set method, which mirrors the behavior of the Tcl set command. To access it during method execution, the instvar method is first used in the method bodies. It is analogous to the export method in CASTE, while no method is used in [incr Tcl]. We chose a declaration scheme because we felt it was important to distinguish instance variables from local variables, much as the global and upvar declarations are used prior to the variables they make accessible. All instance variables and methods are public in the sense of C++ and [incr Tcl] since Object Tcl does not support sealing. We felt that protection was not necessary for well-written programs and found it difficult to reconcile with our goals and Lisp heritage. Protection sometimes interferes with introspection and automatic method combination (discussed later) since the goal of hiding information affects the goals of self-description and flexible code reuse. Figure 2: Superclasses and Method Dispatch (omitted) 4 Inheritance By themselves, objects allow related data and procedures to be grouped. Inheritance mechanisms strengthen this model by allowing the sharing of functionality between different objects, thus encouraging code reuse. Class Safety Safety proc create {acollection} { $self next $acollection $acollection set count 0 } Safety instproc put {thing} { $self instvar count incr count $self next $thing } Safety instproc get {} { $self instvar count if {$count == 0} then { return {} } incr count -1 $self next } Class Stack Stack proc create {astack} { $self next $astack $astack set things {} } Stack instproc put {thing} { $self instvar things set things [concat [list $thing] $things] return $thing } Stack instproc get {} { $self instvar things set top [lindex $things 0] set things [lrange $things 1 end] return $top } Class SafeStack -superclass {Safety Stack SafeStack s s put toast ==> toast s get ==> toast s get ==> {} s destroy ==> {] Figure 3: A class that guarantees safety in a collection and a class that implements a stack collection are mixed to form a safe stack. Figure 4: Class Hierarchy and Ordering for Method Dispatch of a SafeStack (omitted) Inheritance in Object Tcl is based on classes and their superclasses. Each object has a class, and aAfter its own specializations, it inherits functionality from that class and that class's superclasses. Object Tcl supports multiple inheritance --- each class may have many superclasses, and these may be organized in any directed acyclic graph. The class of an object is discovered with the info class method and changed with the class method. Similarly, the superclasses of a class are discovered with the info superclass method and changed with the superclass method. A method is dispatched by searching first the object table of methods, then methods associated with the class, and then up through the superclasses (Figure 2). The first method located is invoked, so methods closer to the object shadow more distant ones. Method dispatch is thus conceptualized as a series of command lookups. The programmer does not explicitly determine the global order in which the superclasses are searched, however. As in CLOS, Object Tcl automatically determines the global order from the local superclass relations. It uses a CLOS-like topological sort algorithm in which direct superclasses of a class take precedence from indirect superclasses. For efficiency, the determined order is then cached since the algorithm's running time is proportional to the size of the superclass graph. Figure 3 shows an example of multiple inheritance. Two classes that adhere to a protocol for managing collections are defined. The Stack class captures the behavior of the astack object that was described previously, allowing many stack objects to be instantiated. The class differs from the object in that the get and put methods are introduced with the instproc method, indicating that they are available for all stack instances, and there is a create method that initializes the internal list for every stack object that is created. The Safety class adds safety checking to collections, keeping track of a count of items and catching attempts to withdraw from empty collections. These classes are combined using multiple inheritance to form a SafeStack class, which is then demonstrated by creating and testing the object s. Figure 4 shows the corresponding class hierarchy and method dispatch. Note that the Safety class can be dynamically combined with any type of collection that adheres to the protocol (for example, stack, queue, and set) but it is not required by any collection. Method Combination The example also demonstrates the use of method combination, where the total behavior of a method is split across classes, rather than simply being shadowed or not. This increases the abstraction capabilities of inheritance, and is found in most advanced object languages. Method combination is the mechanism used to order the constructors and destructors of C++ classes, though it is not recognized by that name. In Object Tcl, method combination is achieved with the next method. When called within a method, it causes the next most shadowed implementation of the method to be invoked. Unlike C++ and [incr Tcl], the next method is not named explicitly, but is determined automatically according to the precedence ordering. This mechanism is similar to around methods in CASTE. Method combination is used in the get and put methods of the Safety mixin class. First, the methods contribute their safety checks, and then the next method is called with next. For the SafeStack class, this ensures that the get and put methods of the Stack class are called, even though the Stack class has no direct relationship to the Safety class. The overall ordering followed in the combination is shown in Figure 4. This style of programming permits the modular combination of methods in an intuitive manner: simply use next, and the system will ensure that the superclass relations hold. In contrast, both C++ and [incr Tcl] require that the next most shadowed method be named explicitly. We believe that this is a major weakness of their multiple inheritance model. It complicates code reuse because it requires programmers to include non-local information about the class hierarchy in method implementations, forcing them to deal with global class orderings. Further, our example would fail in C++ and [incr Tcl] because their inheritance model does not allow calls across the hierarchy other than to superclasses. Table 5: Class Methods (omitted) Figure 5: Class Relationships (omitted) 5 The Class System In Object Tcl, classes are objects themselves, with the added ability to create and manage other objects. They support the superclass relation and serve as a repository for methods on behalf of their objects. This approach lets us apply the power of the object system to influence their behavior. Table 5 lists the methods for class objects. The basic class object that defines the behavior of objects, and from which other classes inherit by default, is Object. Because classes are simply special objects, new classes may be created and destroyed in the same manner as regular objects. Each uses a create method to manufacture new instances, the equivalent of a constructor in C++ and [incr Tcl]. The create method is also the default method, allowing the name of an instance in the method position to cause its creation. Thus, in the examples SafeStack s was interpreted as SafeStack create s, resulting in the creation of a new object. Since classes are themselves objects, they are in turn managed by other classes. These managing objects form a third category of objects called class meta-objects. They are closed under the class relationship, and so are responsible for managing themselves. Figure 5 shows these relationships. The standard class meta-object is called Class. New class meta-objects may be manufactured too, and in conjunction with customizing Object, this allows the behavior of classes to be widely changed. 6 Implementation Object Tcl is available as a Tcl/Tk extension implemented in C. Originally, it was embedded in the VuSystem toolkit but we have recently developed a standalone version for evaluation. It is available via anonymous ftp from ftp.tns.lcs.mit.edu in /pub/otcl. Our implementation goal was to form a compact extension by leveraging the Tcl implementation without changing its core. Methods are implemented in the same manner as Tcl procedures. They are stored in Tcl closure format in an object hash table by their name, and are evaluated by Tcl using its implementation of the proc command, its interpreter, and its calling conventions. Similarly, instance variables are brought into scope using the same mechanism as the upvar and global commands. The result of our efforts is 2000 lines of C code that have little overlap with the Tcl implementation. This is a pleasingly small extension compared to other object extensions and the Tcl core itself. Further, directly using the Tcl implementation of procedures guarantees that Tcl commands that access information further up the call chain (like uplevel and info) will function correctly. The performance cost of our approach also seems reasonable. A basic object method in Object Tcl takes approximately the same time to dispatch as a Tcl procedure with three arguments. A measurement of the time for these operations is 80 and 70 microseconds, respectively. The corresponding measurement for [incr Tcl] is 60 microseconds. For Object Tcl, methods further up the inheritance chain and combined methods may take longer to dispatch, with a time dependent upon the superclass graph. We may revisit them later with the goal of making inheritance and combination as cheap as direct method invocation. However, we feel that absolute performance is less important than our emphasis on compactness and reuse of the Tcl implementation. One interesting aspect of the implementation is the autoloading of Tcl code. Demand loading is important to reduce startup time. It is complicated for object-oriented programming because objects and methods may be referred to in more ways that previously. A class, for example, may refer to other classes as its superclasses, or methods may combine with each other along the inheritance chain. Object Tcl addresses this problem by allowing the user to fill a method with a Tcl code stub that forces its complete loading, and by attempting to load undefined classes as they are referenced. Demand loading at the method level as well as the class level permits a finer-grained distribution of startup time. The implementation also features a C application programmer interface that parallels the Tcl level interface described so far. This allows objects and classes to be created and deleted from C, as well as methods whose bodies are implemented in C to be added to objects. Figure 6: A Motion Segmentation program built with the VuSystem 7 A Visual Programming Example The VuSystem multimedia application of Figure 6 demonstrates the use of Object Tcl. It shows a visual programming interface of a video motion segmentation program. The interface is used to manipulate the program as it runs. The video program is written as an Object Tcl script (using an older embedded version of the language). Code that manipulates the video frames is written in C, while code that controls the user interface is written in Tcl. The program displays the video windows as part of its normal operation, but the graph structure that is drawn on the screen as a program representation is dynamically produced at the user's request. The visual programming interface is constructed by using introspective features to query the program objects for their organization. Each video processing object is first asked what it is connected to in order to enumerate all objects, and then asked to draw a representation of itself on the screen. Most objects draw themselves by inheriting a generic method implementation. They appear as the labelled rectangles linked to each other. Some objects, such as those representing a group of local objects or remotely executing objects, require a special representation. This is accomplished by shadowing the generic method with a specialized implementation. Thus, some of the labelled rectangles in Figure 6 represent a group of processing modules that appear to function as a single larger module. In this manner, the inheritance is used to capture the common patterns while delegation permits special cases to be handled as they arise. Further, the operation polymorphism of object-style naming means that the same code may be used to construct visual programming interfaces for a range of video programs, even if the interfaces appear to be quite different. The dynamic extensibility of Object Tcl is also important to the user interface. Manipulations of the interface are translated into object commands that are evaluated directly, and in this way the running program is reconfigured. Video processing modules may be deleted and new ones created, or their pattern of processing changed. Further, new classes may be manufactured by grouping existing objects. This is analogous to the mega-widgets of [incr Tk]. 8 Future Work We plan to extend Object Tcl in three directions to make it more configurable, efficient, and usable. Meta-object protocols (MOPs) let the language user incrementally modify and extend the language definition. The language then occupies a region of design space, rather than a point, and consequently is more widely applicable. For Object Tcl, we would like to exploit the meta-objects to make object traits such as creation, destruction, method dispatch, and precedence orderings accessible to the user. Standard program browsing tools would make effective use of the introspective abilities of the system. Each object could be described in detail, and the entire class inheritance graph displayed. In conjunction with a MOP, these tools could monitor events such as object creation and destruction and aid in debugging. Optimizations, such as caching method dispatches, may help to boost overall performance. The present implementation achieves relative efficiency through its tight integration with the Tcl implementation, but it could be tuned. 9 Conclusions We feel that the design of Object Tcl has several advantages that differ from other object extensions. Object Tcl has grown out of the object command approach of manipulating compliated data through a single Tcl command. It retains this style of programming by allowing individual objects to be specialized, while supporting the construction of object models without replicating the infrastructure that would otherwise be necessary. More than anything else, we attempted to extend Tcl with a core of object-oriented programming facilities rather than to import a separate object programming language to Tcl. Object Tcl largely maintains the runtime extensibility and introspective features of Tcl, as well as Tcl and Tk syntax plus compatibility with C. We feel this is a good match for an object extension. We also sought an economy of design and implementation. Object Tcl provides few ways of forming expressions, yet it incorporates many of the features found in languages such as Flavors and CLOS. It supports multiple inheritance, method combination, and a metaobject-based class system. By leveraging the Tcl implementation without changing the core, Object Tcl is both compact and tightly integrated with Tcl. At 2000 lines of C code, it is a small extension that is easily managed. By reusing the Tcl implementation with little overlap, a high degree of compatibility with Tcl is achieved. Object Tcl is available for evaluation via anonymous ftp from ftp.tns.lcs.mit.edu in /pub/otcl. References (omitted)