Author:             Ashok P. Nadkarni <apnmbx-public@yahoo.com>
Tcl-Version:        9.1
State:              Draft
Type:               Project
Created:            2025-09-19
Keywords:           Initialization
Tcl-Branch:         tip-732

Abstract

This TIP proposes updating and formalizing the method by which Tcl locates its supporting script libraries.

Background

The Tclsh initialization sequence core wiki page shows the call tree for creation of a Tcl interpreter. At a high level, this two has phases:

Tcl_CreateInterp creates the core components of a Tcl interpreter including commands, globals, and initializing subsystems like I/O.
Tcl_Init locates and executes Tcl's support scripts that sets up package loading, encodings and ancillary commands such as auto_exec, unknown etc.

This TIP is concerned with streamlining this latter step which is detailed below. The motivation for the proposed changes is described in a later section.

Tcl executes the following sequence to run an interpreter pre-initialization script, not to be confused with the application initialization script such as .tclshrc or .wishrc.

(1) If the application has registered a pre-initialization script via the call Tcl_SetPreInitScript that is evaluated to initialize the interpreter. The steps below are skipped.

(2) Otherwise, Tcl looks for a procedure called tclInit already defined by the application in the interpreter after the Tcl_CreateInterp call. If found, the script is run and steps below are skipped.

(3) If both (1) and (2) do not hold, Tcl defines its own tclInit procedure which searches for the init.tcl file in several directories as below and sources it. The location of the init.tcl file is also set as the value of the tcl_library global variable in the interpreter. The init.tcl script does the bulk of the work referenced earlier such as setting up auto_path, unknown, encoding directories etc.

This TIP proposes to change the list of directories that Tcl searches in this last step, the result of which is stored in tcl_library and added to the encoding search path.

In Tcl 9.0, this search is implemented by first constructing a list of directories to search. Note the complete list of candidates is constructed before checking if any particular directory contains init.tcl. Directories are added to search path list in the following order:

(3a) If the application has already set the tcl_library variable, its value is added to the search list. Moreover, the remaining search steps 3b-3j are skipped so this will be the only directory in the search path.

(3b) $env(TCL_LIBRARY), $env(TCL_LIBRARY)/tcl9.0 if env(TCL_LIBRARY) defined and not empty.

(3c) The library directory within the ZipFS archive if present

(3d) $tclDefaultLibrary if defined

(3e) [pgkconfig::get scriptdir,runtime] but only if tclDefaultLibrary was not defined.

(3f) $parent/lib/tcl9.0 where parent is [file dirname [file dirname [info name]]]

(3g) $parent/../lib/tcl9.0

(3h) $parent/library

(3i) $parent/../library

(3j) $parent/../tcl9.0/library

(3k) $parent/../tcl9.0.2/library (patchlevel)

(3l) $parent/../../tcl9.0.2/library (patchlevel)

(3j) Every element of the tcl_libPath list if it is defined

(4) Each directory in the list constructed in (3) is examined for the presence of the init.tcl file. If found, it is sourced. On success, the tcl_library variable is set to the directory path and no more directories are examined. If the init.tcl script raises an error, the iteration continues examinining the remaining directories.

(5) The init.tcl script sets up auto_path, encoding directories, unknown handlers for packages and commands, and the auto exec, auto load and auto indexing facilities.

Motivation

The changes to the current behavior are driven by the following considerations:

User configuration should not overridden. If user specifies a path, and it does not exist, raise a configuration error. Tcl should not try to guess, at least not until TIP 130 is implemented. This is akin to raising an error on invalid encoding and not silently replacing with a guessed encoding. Based on this, handling of env(TCL_LIBRARY) in (3b) should be similar to that of tcl_library in (3a) in that no further directories or subdirectories should be searched. Note this change would in fact match the documented behavior of env(TCL_LIBRARY).
Along the same lines, limit the directories searched. Do not search grandparents and further ancestors. The hunt for a suitable directory can be taken too far and has the potential for security holes. The rationale is the same reason that system loaders restrict directories searched to those explicitly listed. As above, layouts involving grandparents etc. can be supported using pkgconfig.
Mechanisms that are undocumented or duplicated should be removed. Accordingly, eliminate step (2) - check for the application defined tclInit. It is undocumented, not covered by the test suite and there is similar functionality available through Tcl_SetPreInitScript (though it is not exactly the same as it is global and not per interpreter). Similarly, eliminate (3j) - tcl_libPath. Again, undocumented, untested and marked OBSOLETE in the code. Applications should use env(TCL_LIBPATH) or directly set tcl_library instead.
Tcl expects certain setup to be done by the init scripts, in particular, setting of tcl_library and encodings. Check that the application defined scripts have done this or raise an error. This avoids anomalous behaviour later that takes effort to track down.
In (4), once init.tcl is found and evaluated, propagate any exceptions. Do not ignore them and do not continue searching for another init.tcl. To reiterate, misconfiguration should not be ignored and attempts made to "fix it up" leading to unpredictable results. Evaluating init.tcl would have already changed interpreter state.
Eliminate version independent paths that are not explicitly included via configuration or the environment. Thus (3h) and (3i), e.g. $parent/../library, are removed. The latter leads to proven misconfiguration and head scratching as to cause of failures in the presence of multiple Tcl versions.
Remove patchlevel based search. If at all necessary, the same can be effected through the pkgconfig mechanism (3e) above.
Explicitly document search paths. Currently, the manpage documentation is both incomplete and incorrect.
Re-implement the directory search and invocation of init.tcl to from script to C. This is of course solely for faster initialization of interpreters, not for functional reasons. Similarly, move the initialization of auto_path from init.tcl to C. Together these result in an reduction in interpreter initialization time of 25%.

Specification

The resulting initialization sequence with the changes above is shown below:

(1) If the application has registered a pre-initialization script via the call Tcl_SetPreInitScript that is evaluated to initialize the interpreter.

(2) Otherwise, Tcl (via the new LocatePreInitScript internal function) searches for the init.tcl file in several directories as listed in (2a)-(2f).

(2a) If the application has already set the tcl_library variable, the search is ended without checking for the presence of an init.tcl file. This follows 9.0 behavior.

(2b) Likewise, if env(TCL_LIBRARY) is defined, search is ended irrespective of the presence of an init.tcl file. This matches 9.0 documentation, not 9.0 behavior.

(2c) The library directory within the ZipFS archive if present as in 9.0

(2d) $tclDefaultLibrary as in 9.0

(2e) [pgkconfig::get scriptdir,runtime]. This is checked regardless of (2d) unlike 9.0

(2f) $parent/lib/tclVERSION where parent is [file dirname [file dirname [info name]]]

(3) If no directory was found in step (2), an exception is raised. Otherwise the tcl_library variable is set to the found directory.

(4) An attempt is made to source the init.tcl file in the directory. If the file does not exist (this can happen with (2a) and (2b) above) an error is raised. Unlike 9.0, if the found init.tcl raises an exception, it is propagated. In either error case, further directories are not searched.

(5) The init.tcl script calls the tcl::InitAutoPath internal command that sets up auto_path. This command, implemented in C, moved from init.tcl script to append to auto_path the values in env(TCLLIBPATH), tcl_library and its parent, and $parent/lib. In the case of safe interpreters, auto_path is initialized to empty if it did not already exist.

The remaining sections of init.tcl that set up handlers for packages and commands, and the auto exec, auto load and auto indexing facilities, are unchanged.

Implementation notes

Implementation is in the tip-732 branch.

References

TIP 66 describes some aspects of Tcl initialization. While almost a quarter century old, much of its content related to integration with applications still holds.
Tclsh initialization sequence page in the core wiki shows the call tree for tclsh startup in 9.0.2.

Copyright

This document has been placed in the public domain.