Pp. 31–36 of the Proceedings

Charity Telethon Supported by Tcl/Tk

A set of Tcl/Tk Version 8 scripts supports a telethon that raises money for needy families. This paper describes the system and how it was constructed. Because the system uses nearly every major feature of the Tcl and Tk system, including the Tcl web server, this paper serves as an example that emphasizes the power of universal scripting in both the development and deployment of a distributed system.

For the past 19 years in Maynard, Massachusetts, the local high school radio and television station, WAVM, stages a telethon to raise money for a local charity supporting needy families in the area. This annual community event culminates in a 40-hour, non-stop television telethon, which includes live auction segments taking place throughout the event. The telethon weekend is staffed by over 100 students (from 6th through 12th grade), parents, and other community members.

In 1992 a (then) new network and VAX/VMS computer system had been installed at the high school where the telethon originates. The author wrote the first telethon auction support program in some earlier scripting languages: DCL (Digital Command Language) and GNU awk. This program served the telethon well, but was an absolute nightmare to maintain. It was also dependent on hardware (VAX 4000 and VT420 "dumb" terminals) that has a limited future within the school system.

For the 1997 WAVM Telethon, I decided that it was time to free ourselves from the trusty VMS-based system and move to a system that took advantage of the PCs, servers, and network infrastructure that has grown over the past 5 years. WAVM’s Web department was also growing rapidly and, for the first time, taking a major role in the telethon’s presentation to the public. So, we were interested in integrating facets of the auction to an unknown Internet audience.

I was prepared to develop the whole system myself, but a fellow Tcl hacker wanted to learn more about Tk and offered to help. However, his time was limited. This situation was further complicated by his taking a new job at another company–which meant that our communication would be primarily through e-mail. We opted for a simple-but-flexible architecture that takes advantage of the computing infrastructure at the school, as shown in the following diagram:

Even a "midnight" project needs priorities and we set them early on, as follows:

• Support the auction. Provide programs to maintain the inventory of donated items and accept bids made over telephones during the telethon.
• Keep the studios in touch with the auction. The first priority here was a way of letting the telethon hosts know the current top bids.
• Make managing the telethon easier on the studio floor managers and on the adult support team. This includes reports, real-time monitors, etc.
• Integrate the World-Wide Web with telethon activities--particularly the auction. (In parallel with all this work, we were preparing to webcast the full 40-hour telethon live via RealVideo.)

The first component to be constructed was the database server. I initially considered using the AltaVista Forum Toolkit [1], because it had a built-in database manager with journalling, and I was also very familiar with the interface. I did not consider other database packages (for example, MySQL) because I had no prior experience with them (and I didn’t really have time to evaluate and select one). Plus it would require compilation and testing for my target platforms (Intel & Alpha). The AltaVista Forum software is based on Tcl 7.6–another minus. I opted to use a 100% Tcl approach, using the Tcl 8.0 release to construct a simple in-memory database with journalling for reliability [2].

The auction database consisted of three datasets: the items, the callers, and the bids. Each dataset had unique characteristics in terms of access and processing, but they all required common storage management and journalling/recovery features. (A windstorm in 1993 knocked out power during the middle of a telethon, so reliability features were historically mandated.)

Each dataset was given its own Tcl namespace within which to work. Database operations such as Add, Remove, etc., were implemented as Tcl procedures within the namespace. Because the total estimated size of the database was under a megabyte, the data for each dataset was stored in memory (as a Tcl array or list). This decision provided the server with a very high-performance base on which the reliability features were built.

It is interesting to note that while the Items and Callers datasets were implemented as one or more arrays that provided keyed access to the records, the Bids dataset was implemented as a simple Tcl list. Tcl 8 list operations are orders of magnitude more efficient than previous versions, and accessing elements of a list with 1000+ members consumes very little processing time. Based on historical data from the old system, we calculated an upper limit of 2,000 elements for any one list or array. So, we were well within what Tcl could efficiently manage using this simple approach to databases.

A single set of Tcl procedures implemented common operations such as Open, Close, Create Checkpoint, and Recover. These procedures were entered into each dataset’s namespace (eval), where they merged with the other dataset-specific operations. This technique simulates an object-oriented inheritance operation, greatly reducing the amount of code needed for each dataset. The following figure shows all of these common methods as implemented in the database server:

The common methods maintain a so-called .info file for each dataset, which coordinates the versioning of journal and checkpoint files.

The journalling and recovery feature exploits the "data represented as scripts" philosophy [3]. The lowest level database operations are implemented as Tcl procedures. The procedures that modify the database simply "record themselves" in the journal file as a Tcl script, as in the following example:

When the database server shuts down, each data set will save a copy of itself to disk (checkpoint). This is as simple as puts -nonewline $chan [array get items]. When a database is opened, the latest checkpoint file is read into memory. Any transactions in the journal are then replayed by reading in the procedure calls and evaluating them, thus reapplying the missing transactions.

A fourth namespace, called "Tx", holds all the public transaction methods, again implemented as Tcl procedures. A Tcl socket connection manager accepts incoming TCP/IP connections. Upon receipt of a connection it then sets up a fileevent with the Tx::tx procedure called whenever the channel is readable. The database server implements a very simple protocol that allows the movement of arbitrary amounts of data. For both requests and responses, the data length is sent first (terminated by a linefeed), followed by the data.

The entire database server is implemented as an 850-line Tk script, which also provides a logging window for database events, a checkbox to enable debugging instrumentation, and a master switch to enable or disable bidding. The latter is used to assure that none of the bidding stations can accept bids until the auction officially starts. (The author could be seen running across the room in a panic at the start of the auction, because the first phone call had arrived but this "master switch" had not yet been thrown.)

An auction is divided into 3 phases and there is specific software devoted to each phase.

All of these scripts were under the control of an "umbrella" script, which did a simple password check and gave access to a subset of the capabilities as needed. This script would dynamically load (source) in the particular program needed from a master network file share.

Two television studios used during the telethon. We needed a way of relaying current bid information to the on-camera telethon hosts in a smooth and inobtrusive manner regardless of which studio they might be in. During the course of developing this capability it became apparent that a powerful and flexible system could easily be constructed to provide this critical information and potentially much more.

A "message switch" Tk script was designed to run during the entire telethon, much the same way the database server was always available. The message switch would accept messages via the standard socket and fileevent mechanisms. These messages could either listen for messages or leave a message. Every time a message was left, its payload would be the response for any outstanding listen messages. This was done by the listen transactions doing a vwait on the message number, which was incremented by the leave message. The following procs implement the core functions of the message switch:

proc connHandler { chan ipAddr port } {
    global activeChannels
    log conn "Starting channel $chan"
    fconfigure $chan -blocking no
    fileevent $chan readable "request $chan"
    set activeChannels($chan) "open"
}

proc request { chan } {
    global activeChannels
    fconfigure $chan -blocking no
    if {[gets $chan rqst] < 0} {
        fileevent $chan readable ""
        close $chan
        log conn "Closing channel $chan"
        unset activeChannels($chan)
        return
    }
    log debug "Tx $chan: $rqst"
    # Execute transaction (unsafely)
    if {[catch {eval $rqst} err]} {
        response $chan [list ERROR $err]
    }
    catch {flush $chan}
}

proc response { chan data } {
    if {[catch {puts $chan $data} err]} {
        log debug "Response Error: $chan"
    } else {
        log debug "Sent $chan data: $data"
    }
}

proc listen {} {
    upvar chan chan
    global messageNumber
    global lastMessage
    vwait messageNumber
    response $chan $lastMessage
}

proc leave { text } {
    upvar chan chan
    global messageNumber
    global lastMessage
    set lastMessage $text
    incr messageNumber
    response $chan "Message $messageNumber sent"
}

The mechanism was very simple and did not attempt to provide any guarantees regarding message delivery, but it was more than sufficient for the task.

When no messages were received by the teleprompter for approximately a minute, an after event would display a "countdown clock" of how many hours were left in the telethon.

Although providing a Web interface was desirable, it was pretty much last on the priority list. Fortunately, I had the opportunity to be using the Tcl web server [4] in a separate project, so it appeared that integrating an auction inventory query facility would be relatively straightforward.

An interface to the message switch was also created to allow Internet viewers to interact with the students. However, there was insufficient time to fully develop and test the capability. So, we dropped back to using e-mail for feedback.

The telethon would eventually utilize at least thirteen computers deployed in various rooms and studios. Given our limited resources and fixed schedule there was not much time for testing the system in the "all up" configuration. Critical components like the database server were unit tested by test harness scripts, to flush out any bugs and to validate that the performance was acceptable.

Our custom web server was an unexpected bright spot. We really were not sure if the idea of using the Internet for a local event such as ours would attract anyone. During the telethon, just over 7,800 web transactions were processed, with a number of small updates to the program applied on-the-fly and no interruptions in service. Feedback received during the auction was universally positive for this service, and we reached people far outside the normal broadcast range. For the short amount of time it took to make it all work, this was well worth the effort.

The true "script once, execute anywhere" was critical to the timely development of the software. This is highlighted by two observations:

Reuse of existing scripts and code, such as the Tcl web server, made it possible to add new features quickly as they became necessary. The decision to build the database server "from scratch" rather than to use a more traditional database server did not adversely affect the project. An analysis of the database server shows that roughly 325 lines of code actually implement the core database functions; the remaining code would have been necessary even if an SQL database was used instead.

Our telethon raised over $36,700 for local needy families, with the auction bringing in nearly $9,600, both amounts were new all-time records.

With the success of the telethon behind us, we are exploring how to apply this technology to day-to-day station operations. Our first task was to create a way to switch our webcasting services on and off via the web server. For upcoming telethons we will explore increasing the interaction between the web audience and the studios (messages, challenges, etc.), perhaps with limited auctioning occurring on the web itself (without SSL we need to find a way of easily authenticating bidders). One thing is for certain, the WAVM Telethon will continue to be "Tcl Powered".

[1] David Griffin. "Tcl in AltaVista Forum", 5^th Annual Tcl/Tk Workshop Proceedings, 1997

[4] Brent Welch and Steven Uhler. "Web Enabling Applications", 5^th Annual Tcl/Tk Workshop Proceedings, 1997