The Tcl Extension Architecture (TEA)

The Source Distribution

Table See The Tcl source directory structure. describes the directory structure of the Tcl source distribution. The Tk distribution is similar. The directory structure divides the sources into generic and platform-specific directories.

The Installation Directory Structure

When you install Tcl, the files end up in a different arrangement than the one in the source distribution. The standard installation directory is organized so it can be shared by computers with different machine types (e.g., Windows, Linux, and Solaris). The Tcl scripts, include files, and documentation are all in shared directories. The applications and programming libraries (i.e., DLLs) are in platform-specific directories. You can choose where these two groups of files are installed with the --prefix and --exec-prefix options to configure . The --prefix option specifies the root of the installation directory (e.g., /usr/local). The --exec-prefix option specifies a platform-specific directory (e.g., /usr/local/solaris-sparc) for applications and programming libraries. Table See The installation directory structure relative to the --prefix directory. shows the standard installation directory structure:

Binary Library

A binary library (i.e., DLL) goes into the platform-specific lib directory. For example, you install your DLL into exec_prefix/lib/libfoobar1.0.so and you create a package index file in exec_prefix/lib/foobar1.0/pkgIndex.tcl, which contains this code:

package ifneeded foobar 1.0 [list load \

[file join $dir .. \

libfoobar1.0[info sharedlibextension]]\

Foobar]

Collecting all the binary libraries in one directory makes it easy to resolve dependencies among them and third-party libraries that support your Tcl extension. UNIX users may have to adjust their LD_LIBRARY_PATH to include the exec_prefix/lib directory. On Windows, these files are actually in the exec_prefix/bin directory, which is automatically searched. Keeping the pkgIndex.tcl files in separate directories keeps them independent.

Tcl Scripts

If your extension is just Tcl scripts, then it can be shared by users on different platforms. These libraries are typically kept in a subdirectory of prefix/lib, (e.g., /usr/local/lib/foobar1.0). You can use the pkg_mkIndex Tcl command to generate a pkgIndex.tcl file for your scripts:

pkg_mkIndex -verbose prefix/lib *.tcl

By default, pkg_mkIndex generates pkgIndex.tcl files that contain tclPkgSetup commands that use source or load indirectly. You might imagine that package require actually loads code, but by default is does not. Instead, the following tclPkgSetup command arranges for foobar.tcl to be sourced whenever the unknown command tries to find Foobar_Init , Foobar_DoSomething , or Foobar_End .

package ifneeded foobar 1.0 \

[list tclPkgSetup $dir foobar 1.0 \

   {{foobar.tcl source {Foobar_Init

Foobar_DoSomthing Foobar_End}}}]

The tclPkgSetup command is complex, so you should use the pkg_mkIndex command to generate these commands for you. If you use pkg_mkindex -direct , you can create a simpler package that is sourced immediately in response to the package require command. This direct package index looks like this:

package ifneeded foobar 1.0 \

[list source [file join $dir foobar.tcl]]

Library and Script Combination

If you have both scripts and binary libraries, then you can split your package into two parts: the shared part as Tcl scripts, and a platform-specific part as a binary library. The tricky part is building your pkgIndex.tcl file correctly. There are two problems. First, you can only have one package ifneeded command for a single package, so you need to specify something about the scripts and the library in one command. Next, you cannot predict the location of both parts of the package, so you have to assume they are installed in a standard location relative to the auto_path. Our preferred solution is modeled after the one in the SNACK sound extension by Kåre Sjölander.

Create a pkgIndex.tcl file in the exec_prefix/lib/foobar1.0 subdirectory. You will need to install a copy for each different platform that you compile for (e.g., solaris-sparc/lib/foobar1.0/pkgIndex.tcl and linux-ix86/lib/foobar1.0/pkgIndex.tcl.). This file loads the binary library and sources the Tcl script. We assume that the scripts are installed relative to the Tcl script library:

package ifneeded foobarArch 1.0 \

"[list load \

[file join $dir ../libfoobar1.0[info \

sharedlibextension] Foobar] \;

[list source [file join [file dirname \

$tcl_library] \

foobar1.0/foobar.tcl]]"

This example assumes the standard directory structure. It would be more general to search along the auto_path for the foobar1.0 subdirectory and then source its foobar.tcl file. If you have several script files, you can introduce a short procedure to source all of them. Or, you can have two pkgIndex.tcl files and require that your users require both (e.g., foobar and foobarArch). The packages can also require each other. For example:

package ifneeded foobarArch 1.0 \

"[list load \

[file join $dir ../libfoobar1.0[info \

sharedlibextension] Foobar] \;

[list package require foobar 1.0]"

Finally, by using the package unknown hook, you could define and use an alternate package manager. Newsgroup discussions have pointed out that searching for all the pkgIndex.tcl files can be slow on some systems. An alternate package manager could keep a more compact and efficient database, and perhaps have smarts about downloading packages from standard TEA repositories.

Standard configure Flags

Table See Standard configure flags. shows the standard options for Tcl configure scripts. These are implemented by a configure library file ( tcl.m4 ) that you can use in your own configure scripts. The facilities provided by tcl.m4 are described in more detail later. There are also many other command line options that come standard with configure . Some of these are meant to give you control over where the different parts of the installation go. However, because of the way Tcl automatically searches for scripts and binary libraries, you can mess up the Tcl installation by installing the libraries and the binaries in wildly different locations. Because of this, the Tcl installation procedures in the standard Makefile do not support the --libdir and --bindir options. In general, if the flags are not listed in Table See Standard configure flags. , then they are not guaranteed to be supported by the standard Makefile template.

Examples

If you only have one platform, simply run configure in the unix (or win ) directory:

% cd /usr/local/src/tcl8.2/unix

% ./configure flags

Use ./configure to ensure you run the configure script from the current directory. If you build for multiple platforms, create subdirectories of unix and run configure from there. You are free to create the compilation directory anywhere (some prefer to keep all the generated files away from the sources.) Here we just use a subdirectory of the unix directory:

% cd /usr/local/src/tcl8.2/unix

% mkdir solaris

% cd solaris

% ../ configure flags

Any flag with disable or enable in its name can be inverted. Table See Standard configure flags. lists the non-default setting, however, so you can just leave the flag out to turn it off. For example, when building Tcl on Solaris with the gcc compiler, shared libraries, debugging symbols, and threading support turned on, use this command:

configure --prefix=/home/welch/install \

--exec-prefix=/home/welch/install/solaris \

--enable-gcc --enable-threads --enable-symbols

Your builds will go the most smoothly if you organize all your sources under a common directory. In this case, you should be able to specify the same configure flags for Tcl and all the other extensions you will compile. In particular, you must use the same --prefix and --exec-prefix so everything gets installed together.

If you use alternate build directories, like the unix/solaris example above, you must specify - -with-tcl when building your extensions. This is the directory where the Tcl build occurred. It contains libraries and a tclConfig.sh file that is used by the extensions configure process.

If your source tree is not adjacent to the Tcl source tree, then you must use --with-tclinclude or --with-tcllib so the header files and runtime library can be found during compilation. Typically this can happen if you build an extension under your home directory, but you are using a copy of Tcl that has been installed by your system administrator. The --with-x-includes and --with-x-libraries flags are similar options necessary when building Tk if your X11 installation is in a non-standard location.

Finding a working compiler

As the configure script executes, it prints out messages about the properties of the current platform. You can tell if you are in trouble if the output contains either of these messages:

checking for cross compiler ... yes

or

checking if compiler works ... no

Either of these means configure has failed to find a working compiler. In the first case, it assumes you are configuring on the target system but will cross-compile from a different system. Configure proceeds bravely ahead, but the resulting Makefile is useless. While cross-compiling is common on embedded processors, it is rarely necessary on UNIX and Windows. The cross-compiling message typically occurs when your UNIX environment isn't set up right to find the compiler.

On Windows there is a more explicit compiler check, and configure exits if it cannot find the compiler. Currently, the Windows configure macros knows only about the Visual C++ compiler. VC++ ships with a batch file, vcvars32.bat , that sets up the environment so you can run the compiler, cl , from the command line. You must run vcvars32.bat before running configure , or set up your environment so you do not have to remember to run the batch file all the time.

Installation Directories

The --prefix flag specifies the main installation directory (e.g., /home/welch/install ). The directories listed in Table See The installation directory structure relative to the --prefix directory. are created under this directory. If you do not specify --exec-prefix , then the platform-specific binary files are mixed into the main bin and lib directories. For example, the tclsh8.2 program and libtcl8.2.so shared library will be installed in:

/home/welch/install/bin/tclsh8.2

/home/welch/install/lib/libtclsh8.2.so

The script libraries and manual pages will be installed in:

/home/welch/install/lib/tcl8.2/

/home/welch/install/man/

If you want to have installations for several different platforms, then specify an --exec-prefix that is different for each platform. For example, if you use --exec-prefix=/home/welch/install/solaris , then the tclsh8.2 program and libtcl8.2.so shared library will be installed in:

/home/welch/install/solaris/bin/tclsh8.2

/home/welch/install/solaris/lib/libtclsh8.2.so

The script libraries and manual pages will remain where they are, so they are shared by all platforms. Note that Windows has a slightly different installation location for binary libraries (i.e., DLLs). They go into the exec_prefix /bin directory along with the main executable programs.:

configure.in

The configure.in file is the template for the configure script. This file is very well commented. The places you need to change are marked with __CHANGE__ . The first macro to change is:

AC_INIT(exampleA.h)

The AC_INIT macro lists a file that is part of the distribution. The name is relative to the configure.in file. Other possibilities include ../generic/tcl.h or src/mylib.h , depending on where the configure.in file is relative to your sources. The AC_INIT macro is necessary to support building the package in different directories (e.g., either tcl8.2/unix or tcl8.2/unix/solaris ).

The next thing in configure.in is a set of variable assignments that define the package's name and version number:

PACKAGE = exampleA

MAJOR_VERSION = 0

MINOR_VERSION = 2

PATCH_LEVEL =

The package name determines the file names used for the directory and the binary library file created by the Makefile. This name is also used in several configure and Makefile variables. You will need to change all references to "exampleA" to match the name you choose for your package.

The version and patch level support a three-level scheme, but you can leave the patch level empty for two-level versions like 0.2 . If you do specify a patch-level, you need to include a leading "." or "p" in it. These values are combined to create the version number like this:

VERSION = ${MAJOR_VERSION}.${MINOR_VERSION}${PATCH_LEVEL}

Windows compilers create a special case for shared libraries (i.e., DLLs). When you compile the library itself, you need to declare its functions one way. When you compile code that uses the library, you need to declare its functions another way. This complicates the exampleA.h header file. Happily, the complexity is hidden inside some macros. The standard configure.in defines a build_ Package variable with the following line, which you do not need to change:

AC_DEFINE_UNQUOTED(BUILD_${PACKAGE})

The build_packageA variable is only set when you are building the library itself, and it is only defined when compiling on Windows. We will show later how this is used in exampleA.h to control the definition of the Examplea_Init procedure.

The configure.in file has a bunch of magic to determine the name of the shared library file (e.g., packageA02.dll , packageA.0.2.so , packageA.0.2.shlib , etc.). All you need to do is change one macro to match your package name.

AC_SUBST(exampleA_LIB_FILE)

These should be the only places you need to edit when adapting the sample configure.in to your extension.

Makefile.in

The Makefile.in template is converted by the configure script into the Makefile . The sample Makefile.in is well commented so that it is easy to see where to make changes. There are a few variables with exampleA in their name. In particular, exampleA_LIB_FILE corresponds to a variable name in the configure script. You need to change both files consistently. Some of the lines you need to change are shown below:

exampleA_LIB_FILE = @exampleA_LIB_FILE@

lib_BINARIES = $(exampleA_LIB_FILE)

$(exampleA_LIB_FILE)_OBJECTS = $(exampleA_OBJECTS)

You must define the set of source files and the corresponding object files that are part of the library. In the sample, exampleA.c implements the core of the Secure Hash Algorithm, and the tclexampleA.c file implements the Tcl command interface:

exampleA_SOURCES = exampleA.c tclexampleA.c

SOURCES = $(exampleA_SOURCES)

The object file definitions use the OBJEXT variable that is .o for UNIX and .obj for Windows:

exampleA_OBJECTS = exampleA.${OBJEXT} tclexampleA.${OBJEXT}

OBJECTS = $(exampleA_OBJECTS)

The header files that you want to have installed are assigned to the GENERIC_HDRS variable. The srcdir Make variable is defined during configure to be the name of the directory containing the file named in the AC_INIT macro:

GENERIC_HDRS = $(srcdir)/exampleA.h

Unfortunately, you must specify explicit rules for each C source file. The VPATH mechanism is not reliable enough to find the correct source files reliably. The configure script uses AC_INIT to locate source files, and you create rules that use the resulting $(srcdir) value. The rules look like this:

exampleA.$(OBJEXT) : $(srcdir)/exampleA.c

$(COMPILE) -c '@CYGPATH@ $(srcdir)/exampleA.c' -o $@

The cygpath program converts file names to different formats required by different tools on Windows. On UNIX, the CYGWIN macro is simply defined to echo .

Standard Header Files

This section explains a technique you should use to get symbols defined properly in your binary library. The issue is raised by Windows compilers, which have a notion of explicitly importing and exporting symbols. When you build a library you export symbols. When you link against a library, you import symbols. The BUILD_exampleA variable is defined on Windows when you are building the library. This variable should be undefined on UNIX, which does not have this issue. Your header file uses this variable like this:

#ifdef BUILD_exampleA

#undef TCL_STORAGE_CLASS

#define TCL_STORAGE_CLASS DLLEXPORT

#endif

The TCL_STORAGE_CLASS variable is used in the definition of the EXTERN macro. You must use EXTERN before the prototype for any function you want to export from your library:

EXTERN int Examplea_Init _ANSI_ARGS_((Tcl_Interp *Interp));

The _ANSI_ARGS_ macro is used to guard against old C compilers that do not tolerate function prototypes.

Using the Sample Extension

You should be able to configure , compile and install the sample extension without modification. On my Solaris machine it creates a binary library named exampleA0.2.so , while on my Windows NT machine the library is named exampleA02.dll . The package name is Tclsha1 , and it implements the sha1 Tcl command. Ordinarily these names would be more consistent with the file names and package names in the template files. However, the names in the sample are designed to be easy to edit in the template. Assuming you use make install to copy the binary library into the standard location for your site, you can use the package from Tcl like this:

package require Tclsha1

sha1 -string "some string"

The sha1 command returns a 128 bit encoded hash function of the input string. There are a number of options to sha1 you can learn about by reading the man page that is part of the sample.