TIP 477: Modernize the nmake build system

Login
Home Timeline Branches Tags Tickets Help
Bounty program for improvements to Tcl and certain Tcl packages. 
Author:         Ashok P. Nadkarni <apnmbx-wits@yahoo.com>
State:          Final
Type:           Project
Vote:           Done
Created:        30-Sep-2017
Post-History:
Keywords:       Windows nmake build
Tcl-Version:    8.6.8

Abstract

Tcl, Tk and extensions are currently built using one of two build systems - the Tcl Extension Architecture (TEA) and the Windows/Visual C++ specific nmake based environment. This TIP addresses only the latter with a view to modernize it to remove obsolete features, non-optimal build settings, and simplify writing and maintenance of extensions.

This TIP also serves as the documentation of the reworked nmake build system.

Background and Rationale

The current nmake-based build system is based on three files in the win directory of a project.

  • makefile.vc is the master makefile for the project

  • rules.vc is included by makefile.vc and contains the common nmake macro definitions and parsing of configuration options shared by all projects.

  • nmakehlp.c is a helper application that strives to overcome the extremely primitive functionality of nmake. It is invoked from rules.vc to check for supported compiler options, simple string parsing etc.

As currently implemented, the system has several drawbacks including duplication of code making maintenance difficult, unnecessarily verbose extension makefiles, and inconsistent application of compiler options.

Simplifying extension makefiles

Creating a makefile.vc for even a simple extension involves a lot of unnecessary boilerplate leading to copy-n-paste-itis and attendant problems. This TIP aims to simplify the writing of extension makefiles to a minimalist form that reduces the boilerplate to extension-specific configuration.

The following makefile should suffice for the majority of extensions.

PROJECT=sample
!include "rules-ext.vc"
PRJ_OBJS = $(TMP_DIR)\sample.obj \
               $(TMP_DIR)\util.obj
!include "targets.vc"
pkgindex: default-pkgindex

(In contrast, the current makefile for the sample extension is of the order of 300 lines, exclusive of comments.)

In addition to simplification of the makefile itself, the TIP also proposes removing the burden from the extension author of having to write their own Windows version resource file, pkgIndex.tcl and other common boilerplate.

For more complex extensions, the build system should be incrementally customizable using standard "building block" macros.

Ensuring consistency

In principle, rules.vc and nmakehlp.c should be shared across Tcl, Tk and all extensions. In practice, each has its own copy leading to divergence between the various copies and the resulting maintenance headaches. At the time of writing, every single extension, including Tk, had diverged from Tcl. This means updates for new compilers, changes in Tcl build configuration (e.g. -DUNICODE) do not make their way into extensions. Moreover, making fixes involves individually fixing extensions, some of which are orphaned.

The TIP proposes distribution of the nmake support files so that each extension, including Tk, is built off Tcl's master copy (either installed or from source) without having to maintain its own.

This not only ensures consistency but also reduces the burden on extension authors to maintain their makefiles to keep up with Tcl and Microsoft compiler changes.

Auditing compiler configuration

There are some bugs, inconsistencies and misconfiguration of compiler options in the various incarnations of the extension makefiles. Examples include unoptimized release builds in some important extensions like Tk and Sqlite, differing floating point conformance options in debug and release builds even within Tcl and so on.

Therefore, a standard set of documented nmake macros are defined to ensure consistency across build configurations.

Non-goals

It is not the intent of this TIP to look at alternatives to nmake. There is already the TEA based system for those who prefer to use it. Sean Woods is also working on the practcl build tools which may supplant both TEA and nmake in the future.

The nmake build system is not intended to be "compatible" with the TEA based system. For example, the generated libraries may not link with those generated from the other system. Of course, the built extensions should run with Tcl compiled from any compiler if stubs are enabled. At the same time, the modernized nmake system does make some minimal integration with TEA so that configuration values such as version numbers, pkgIndex.tcl.in etc. only need to be defined in one place.

Certain limitations in the current nmake system, in particular the requirement that there be no spaces in the source directories path are not addressed.

Command line interface

There will be no changes to the command line interface used to build Tcl and extensions using nmake except for removal of some obsolete options and some minor additions.

nmake /f makefile.vc ?targets? ?macros?

The targets are as always defined by the makefile. However, the system also has standard predefined targets that makes it unnecessary for the extension makefile to define targets in many common cases. These are detailed later.

Build configuration macros

The macros passed on the command line are for the most part the same as in the current system. They specify the build configuration such as whether an extension is built as a static or shared library, instrumentation, generation of debug information and so on.

Here we describe the macros that can be specifed for Tcl and all extensions. An extension may define additional macros as well as additional values for the macros defined here.

Directory paths INSTALLDIR, TMP_DIR, OUT_DIR

The installation directory is specified via the INSTALLDIR macro on the command line. If unspecified, this defaults to C:\Tcl. NOTE: this is a change from the current default of C:\Program Files\Tcl because the latter may not have access permissions for the user.

The OUT_DIR macro specifies the directory in which to place the output executables and libraries while TMP_DIR specifies the directory for object files. If unspecified, the defaults used are based on the options specified on the command line, compiler version and target architecture. The output directory path placed in OUT_DIR has the form Release_COMPILERVERSION for x86 release builds and Release_AMD64_COMPILERVERSION for x64 release builds (the discrepancy in inclusion of architecture is historical). For debug builds (when symbols is included in OPTS on the command line), Release_ is replaced by Debug_. The object files directory path placed in TMP_DIR has the form $(PROJECT)_ThreadedDynamic or $(PROJECT)_ThreadedStatic depending on whether the build options specify dynamic or static linking.

The OPTS macro

The OPTS macro controls the build configuration for the compiler and linker. Its value is a comma-separated list of option values, the more common options being listed in the table below.

Option Effect
none Nullifies other options even if they are specified.
static Builds the module as a static library instead of the default shared library.
pdbs Generates PDB files with symbol information even for release builds.
symbols Builds a debug version (no optimization, debug C runtime, PDB's).
staticpkg Specifies registry and dde extensions should be statically bound (tclsh and wish only).
msvcrt Links against the dynamic C runtime (see below).
nomsvcrt Links against the static C runtime (see below). Overrides msvcrt.
profile Produces an image that can be used with the Windows Performance Tools profiler.
pgi Instruments the image for profile guided optimization.
pgo Enables profile guided optimization.
nostubs Specifies that extensions should not link against the Tcl stubs library.
unchecked Links the debug build when symbols is specified against the non-debug runtime.
noconfigchecks Disables the checks that ensure that an extension build options are compatible with those of the Tcl against which it is being built.

The msvcrt and nomsvcrt options control which C runtime library is passed to the link step. By default, shared library builds of Tcl link against the C runtime DLL while the static Tcl builds link against the static C runtime library. Specifying the nomsvcrt option will link the shared Tcl library builds to link against the static C runtime. Similarly, specifying the msvcrt option will cause the static Tcl library to link against the C runtime DLL.

The STATS macro

The STATS macro, also specified as a comma-separated list, controls generation of instrumentation code as shown below. This is relevant only for building Tcl itself, not extensions.

Option Effect
none Turns off all instrumentation irrespective of other options being specified.
memdbg Enables instrumentation of memory allocation.
compdbg Enables byte compiler logging for debugging purposes.

The CHECKS macro

The CHECKS macro configure additional compiler checks and warnings.

Option Effect
none Turns off other CHECKS options even if specified.
nodep Disables support for deprecated functions.
fullwarn Cranks up the compiler warnings level.
64bit Enables 64-bit portability warnings.

The following command will generate a static library with PDB debug information, memory instrumentation, full warnings and disabling of deprecated functions.

nmake /f makefile.vc OPTS=static,pdbs STATS=memdbg CHECKS=nodep,fullwarn

The TESTPAT macro

The TESTPAT macro specifies the name of the file containing the tests to be run. This is used by the standard test target described later.

Basic makefiles

Makefile for a basic Tcl extension

NOTE: By convention, makefiles using the nmake system are named makefile.vc. Here we refer to them simply as makefile.

In the simplest case, a Tcl extension follows the common convention where sources are stored in the subdirectories generic, win and compat (not necessarily all). For such an extension, the following serves as a complete makefile.

PROJECT=sample
!include "rules-ext.vc"
PRJ_OBJS = $(TMP_DIR)\sample.obj \
               $(TMP_DIR)\util.obj \
!endif
!include "targets.vc"
pkgindex: default-pkgindex

The lines must be in the order shown with PROJECT defined before inclusion of rules-ext.vc and PRJ_OBJS after. The standard targets, defined in targets.vc, are included last.

Given the above, the commands

nmake /f makefile.vc INSTALLDIR=/path/to/tcl
nmake /f makefile.vc INSTALLDIR=/path/to/tcl OPTS=static
nmake /f makefile.vc INSTALLDIR=/path/to/tcl OPTS=debug
nmake /f makefile.vc INSTALLDIR=/path/to/tcl OPTS=static,debug

will

  • build shared, static and debug versions of the extension depending on the specified OPTS macro

  • create an appropriate pkgIndex.tcl file

  • generate and embed Windows version resource in the binaries.

Note the extension author need not write pkgIndex.tcl or Windows resource definition file unless there is some custom need.

In addition, standard targets for installation and clean up are also included so for example

nmake /f makefile.vc INSTALLDIR=/path/to/tcl install

will install the extension while

nmake /f makefile.vc INSTALLDIR=/path/to/tcl clean
nmake /f makefile.vc INSTALLDIR=/path/to/tcl realclean

will do various levels of cleaning.

The macros that need to be defined for a basic extension are shown below.

Macro Description
PROJECT Name of the package. Must be defined before including rules-ext.vc.
PRJ_OBJS List of object and resource files for building the extension. The object files must be prefixed with $(TMP_DIR)\ as shown above.
PRJ_PACKAGE_TCLNAME (optional)Name for the package require command. Defaults to $(PROJECT)

The PROJECT macro is used to generate the name of the extension binaries, directories etc. On the other hand, PRJ_PACKAGE_TCLNAME is the name of the package as known to Tcl. In most cases, this defaults to the value of $(PROJECT) and need not be specified by the extension makefile. In some cases, where the package names is not a simple ascii string, the two differ, for example, PROJECT may be defined as tkimgbmp while PRJ_PACKAGE_TCLNAME may be defined as img::bmp.

Makefile for a basic Tk extension

In case of a Tk extension, the only change required is to set the PROJECT_REQUIRES_TK macro to 1 before including rules-ext.vc.

PROJECT=sample
PROJECT_REQUIRES_TK = 1
!include "rules-ext.vc"
PRJ_OBJS = $(TMP_DIR)\sample.obj \
               $(TMP_DIR)\util.obj \
!endif
!include "targets.vc"
pkgindex: default-pkgindex

All else remains the same as for a Tcl extension.

The nmake build environment

The nmake build environment described above is implemented through four files rules-ext.vc, rules.vc, targets.vc and nmakehlp.c. The role of each is described in this section.

The rules-ext.vc file

The rules-ext.vc file is intended to be included by the extension's makefile to locate and load the latest compatible rules.vc file. It checks if the installed Tcl has copies of rules.vc and nmakehlp.c that are newer versions than the ones in the extension sources, and if so uses them instead of the extension's copies. In the case of extensions that build against the Tcl source (as opposed to a Tcl installation), it checks the versions in the Tcl source directory in a similar manner.

The compilation rules are versioned via the RULES_VERSION_MAJOR and RULES_VERSION_MINOR macros defined in rules.vc. Versioning is similar to Tcl's in how major and minor versions are treated. When comparing versions, the files in the Tcl installation are used if they have the same major version as that in the extension's rules file and their minor version is the equal or greater.

The nmakehlp.c program

The nmakehlp.c program has the same purpose and functionality as in the current system. It is unchanged and not detailed here. It is compiled and invoked on the fly from nmake for some utility purposes such as extracting versions, searching for strings etc.

The rules.vc file

This is the heart of the current nmake system and remains so, with enhancements to include as much of project-independent functionality as possible. The file is responsible for

  • Determining the compiler environment including target architecture, supported compiler switches etc.

  • Parsing any options and macros supplied by the user on the command line

  • Extracting version numbers, include paths etc.

  • Defining compiler and linker switches, output paths, and standard targets based on the above.

It is intended that there will be only one "master" rules.vc file, the one in the Tcl repository where all changes are made. Extensions will have unmodified copies of this if they need to be build against older versions of Tcl. Otherwise, they will use the one installed with Tcl or from the Tcl sources if building against the latter.

The targets.vc file

This file, optionally included by the extension's master makefile, defines some standard targets that relieves the extension from having to define its own. It is separated from rules.vc so as to permit master makefile to modify macros set by rules.vc before they are expanded in the target rules.

Inclusion of this file is optional in the sense that more complex makefiles may choose to define their own standard targets.

Distributing the nmake build system

To eliminate the issue of divergence between the nmake support files as well as the need for continual maintenance and update, the files rules.vc, nmakehlp.c and targets.vc will be installed as part of a Tcl install in a similar fashion to tclconfig.sh, tclstub86.lib etc. except that they will be placed in the lib\nmake subdirectory under the Tcl installation's root directory.

These files will also be copied to each extension's source repository as is (supposed to be) done today. However, this is only a one-time copy and unlike the current system, it is not required to be done every time Tcl's version of these files change. This also allows the extension to be built against older versions of Tcl that do not include these files in their installation.

Locating headers and libraries

Building an extension requires Tcl, and optionally Tk, header files and libraries.

Locating Tcl

Tcl extensions need to locate either Tcl installed headers and libraries or the Tcl source directory. The nmake build system locates the Tcl directory containing these by trying directories in the following order:

The location of the Tcl header files and libraries is specified with the TCLDIR macro.

  • If the macro TCLDIR is defined on the command line, it is used as the location of the Tcl root directory. This is generally the directory containing an installed Tcl but could also point to the root of a Tcl source tree if the extension requires Tcl internal headers (not recommended).

  • If TCLDIR is not defined, the macro INSTALLDIR is treated as the Tcl root directory if the directory exists and contains the expected header files.

  • As a last resort, the directory ../../tcl is checked and if containing the expected header files, used as the value of TCLDIR.

  • If the header files still cannot be located, an error is generated.

The macros _INSTALLDIR and _TCLDIR are generated from INSTALLDIR and TCLDIR respectively and contain the native form of the path (using backslashes as directory separators).

In addition, the following macros are defined for use by the extension makefile if desired.

Macro Description
TCLINSTALL In the case of extensions, the macro TCLINSTALL is set to 1 if the extension is being built against an installed Tcl and 0 if it is built against Tcl sources.
TCL_MAJOR_VERSION The major version of the Tcl against which an extension is being built.
TCL_MINOR_VERSION The minor version of the Tcl against which an extension is being built.
TCL_PATCH_LEVEL The patch level of the Tcl against which an extension is being built.
TCL_DOTVERSION $(TCL_MAJOR_VERSION).$(TCL_MINOR_VERSION)
TCL_VERSION $(TCL_MAJOR_VERSION)$(TCL_MINOR_VERSION)
_TCL_H The path to the tcl.h header file.

Locating Tk

A similar method as above is used for extensions that build against Tk. The nmake build system locates directories in the following order:

The installation directory is specified via the INSTALLDIR macro on the command line. If unspecified, this defaults to C:\Tcl. NOTE: this is a change from the current default of C:\Program Files\Tcl because the latter may not have access permissions for the user.

The location of the Tk header files and libraries is specified with the TKDIR macro.

  • If the macro is defined on the command line, it is used as the location of the Tk root directory. This is generally the directory containing an installed Tk but could also point to the root of a Tk source tree if the extension requires Tk internal headers (not recommended).

  • If TKDIR is not defined, the macro INSTALLDIR is treated as the Tk root directory if the directory exists and contains the expected header files.

  • As a last resort, the directory $(_TCLDIR) is checked and if containing the expected header files, used as the value of TKDIR.

  • If the header files still cannot be located, an error is generated.

The macro _TKDIR contains the native form of $(TKDIR).

In addition, the following macros are defined for use by the extension makefile if desired.

Macro Description
TKINSTALL In the case of extensions, the macro TKINSTALL is set to 1 if the extension is being built against an installed Tk and 0 if it is built against Tk sources.
TK_MAJOR_VERSION The major version of the Tk against which an extension is being built.
TK_MINOR_VERSION The minor version of the Tk against which an extension is being built.
TK_PATCH_LEVEL The patch level of the Tk against which an extension is being built.
TK_DOTVERSION $(TK_MAJOR_VERSION).$(TK_MINOR_VERSION)
TK_VERSION $(TK_MAJOR_VERSION)$(TK_MINOR_VERSION)
_TK_H The path to the tk.h header file.

Customizing and extending the build environment

It is hoped that the basic makefile structure described earlier will suffice for the majority of extensions which follow standard conventions of source directory location etc. However, extensions may diverge from convention for several reasons.

  • The source subdirectories may not be named generic, compat etc.

  • Additional options, include paths, libraries etc. may need to be passed to the compiler and / or linker

  • The makefile may build multiple extensions and programs. The author does not recommend this as it has few advantages over independent makefiles.

  • It may need customized pkgIndex.tcl or Windows version resources.

  • It may provide its own stubs interface.

  • The installation may need added functionality such as installation of documentation, demo applications etc.

  • And so on.

These all require different levels of customization and extension and the nmake system tries to accomodate these with minimal effort as described in the following sections.

Location of extension sources and implicit rules

The macro ROOT is set by the rules.vc to point to the root of the extension source tree. This must be exactly one level above the directory containing the extension makefile.

The basic makefile shown earlier assumes the following directory structure for the extension sources under $(ROOT):

Subdirectory Macro Description
generic GENERICDIR Directory containing platform-neutral source files.
win WINDIR Directory containing Windows specific source files.
compat COMPATDIR Directory with additional source files implementing functions not present on all platforms.
doc DOCDIR Directory containing documentation files.
demos DEMODIR Directory containing any demo files.
tools TOOLSDIR Directory containing any build tools.
tests TESTDIR Directory containing the test suite

The macros shown in the table above are intialized to the corresponding subdirectory names if not already defined by the parent makefile. Implicit rules are defined to generate the object files corresponding to sources in any of these directories. If the source directories are named differently, the corresponding macro can be defined appropriately before including rules-ext.vc.

So for example, if your generic sources were in directory src, the makefile would be modified as

PROJECT = sample
GENERICDIR = ..\src
!include "rules-ext.vc"
...rest remains same...

In case there are additional directories containing sources, you will have to define an additional implicit rule for each such directory. For example, if the directory extrasrc contained the additional sources, you would add the lines below after including rules-ext.vc.

EXTRADIR = $(ROOT)\extrasrc
...
{$(EXTRADIR)}.c{$(TMP_DIR)}.obj::
        $(cc32) $(pkgcflags) -Fo$(TMP_DIR)\ @<<
$<
<<

Here cc32 and pkgcflags are standard macros defined within rules.vc. These are described later.

Output directories and file names

In addition, rules.vc defines the macros shown below for the output files themselves. These are only generally needed by the extension makefile if it is defining custom targets and rules instead of using the default built-in ones.

Macro Description Example
PRJLIBNAME File name of the generated binary extension sample12t.dll
PRJLIB Path to the generated binary extension .\Release\sample12t.dll
PRJIMPLIB Path to the link library for the generated extension .\Release\sample12t.lib
PRJSTUBLIBNAME File name of the stub library for the extension samplestub12.dll
PRJSTUBLIB Path to the stub library .\Release\samplestub12.dll

The generated file names follow a convention based on the project name (sample), the version (1.2) and a suffix composed of one or more of the letters shown below.

Suffix Meaning
t Threaded build. Default on Windows.
s Static build. Not present for dynamic builds.
g Debug build with symbols.
x Static build but linking to dynamic C runtime.

For most extension makefiles, these file names are not very relevant. However, they are needed when the default rules are not adequate, and the extension makefile needs to define its own.

Build rules

Standard build rules

The nmake build system predefines some standard implicit rules. These rules will search the $(ROOT), $(WINDIR), $(GENERICDIR) and $(COMPATDIR) directories for a source file corresponding to an object file and compile it with the flags appropriate for the current build configuration.

Similarly, a Windows version resource is compiled using the resource compiler by looking in directories $(RCDIR), $(WINDIR) for a resource definition file and generating one if necessary. This is described in a later section.

These implicit rules are defined in rules.vc (included indirectly through rules-ext.vc) and can be disabled for whatever reason by setting the macro DISABLE_IMPLICIT_RULES to 1 before inclusion of rules-ext.vc.

Passing additional compiler switches

By default, the constructed compiler flags include compiler include paths and preprocessor definitions common to all extensions. An extension may wish to extend these without having to define its own rules. This can be accomplished by defining the following macros before including rules-ext.vc.

Macro Description
PRJ_DEFINES Additional preprocessor defines of the form -DMACRO=VALUE
PRJ_INCLUDES Additional directories for locating C header files of the form -IDIRECTORY

Similarly, if the extension requires additional libraries, they can be specified by defining the PRJ_LIBS macro before including rules-ext.vc.

For example,

...
PRJ_DEFINES = -D_CRT_NONSTDC_NO_DEPRECATE -DHAVE_GUMBO
PRJ_INCLUDE = -I"$(ROOT)\expat"
PRJ_LIBS = secur32.lib powerprf.lib
!include "rules-ext.vc"
...

Defining custom build rules

At times, the implicit rules described in the previous section may not meet the needs of an extension. For example, some source files may be located in a non-standard directory, or some additional compiler switches are needed only for a specific file etc. In this case, the makefile may define its own implicit and explicit rules if needed.

The rules.vc file defines some compilation and link macros that make it easier to consistently define custom build rules. It is strongly recommended that if extension makefiles need to define their own implicit or explicit rules, they make use of these macros.

Macros for compilation

An explicit rule may be needed when an object file is to be compiled as an application, not a library. The predefined implicit rules assume the latter.

$(TMP_DIR)\tclWinTest.obj: $(WINDIR)\tclWinTest.c
    $(CCAPPCMD) $?

The CCAPPCMD macro invokes the compiler with all appropriate flags for an application object file based on the build configuration and places it in $(TMP_DIR). The CCPKGCMD macro is similar except it is targeted toward compiling to objects that make up a binary extension, either as a static library or a DLL, based on the build configuration. Finally, the CCSTUBSCMD macro is intended for compiling objects for stubs libraries in cases where the extension implements its own stubs interface.

Another case that requires an explicit rule is when the object file name does not match the name of the source file. In this case, the CC* macros cannot be used either because they also assume the object file name to be derived from the source file name. In this case, we have to drop down to lower level macros.

$(TMP_DIR)\tclMain2.obj: $(GENERICDIR)\tclMain.c
        $(cc32) $(pkgcflags) -DTCL_ASCII_MAIN -Fo$@ $?

In the above example, the object file name does not match the source file and requires an explicit rule where we make use of the cc32 macro which defines the compiler in use and pkgcflags which encapsulates the compiler switches for the build configuration. The rule also happens to define a C preprocessor macro specific to this object file. Note pkgcflags will already include the standard C preprocessor definitions as well as the ones in PRJ_DEFINES.

The pkgcflags macro and similar for other build targets are shown in the table below. All of these are defined in rules.vc and expand to the appropriate compiler switches and flags based on the build configuration and the intended use (application object files, DLL object files etc.)

Macro Description
pkgcflags Compiler switches required to compile a file for an extension that will be linked against the Tcl with or without stubs depending on whether the nostubs option is in effect.
pkgcflags_nostubs Compiler switches required to compile a file for an extension that will be linked against the Tcl library without using stubs.
appcflags Compiler switches required to compile a file for an application program that will be linked against the Tcl with or without stubs depending on whether the nostubs option is in effect.
appcflags_nostubs Compiler switches required to compile a file for an application program that will be linked against the Tcl library without using stubs.
stubscflags Compiler switches that will be used to compile objects for a extension stubs library.

The appcflags and appcflags_nostubs macro includes all switches to be passed to the compiler to compile a source file to an object file to be linked into an application program. These include switches related to optimizations, debug information, warning levels, C runtime selection, C header include paths and C preprocessor definitions.

The pkgcflags and pkgcflags_nostubs are similar except that they include additional C preprocessor definitions required by convention to build a binary extension. These definitions are PACKAGE_NAME, defined as the name of the package, PACKAGE_VERSION, set to the version of the pacakge, MODULE_SCOPE and $(PROJECT)_build. These last two are used in the C source to mark DLL visibility of functions and to distinguish between definition and use of declaration within a C header. A further explanation is out of the scope of the nmake build system.

The last entry in the table above, stubscflags is used to compile an extension's stubs library, if it provides one (most do not).

Macros for linking

Similar to compiler switches, rules.vc defines standard macros, shown in the table below, for linking object files and making libraries.

Macro Description
DLLCMD Macro to link object files into a DLL
LIBCMD Macro to create a static library from object files
CONEXECMD Macro to link object files into a console program
GUIEXECMD Macro to link object files into a GUI program

Just like the compiler macros, these take into account the build configuration (release/debug, shared/static etc.). Here is a fragment from Tcl's makefile that illustrates their use.

!if $(STATIC_BUILD)
$(TCLDDELIB): $(TMP_DIR)\tclWinDde.obj
    $(LIBCMD) $**
!else
$(TCLDDELIB): $(TMP_DIR)\tclWinDde.obj $(TCLSTUBLIB)
    $(DLLCMD) $**
    $(_VC_MANIFEST_EMBED_DLL)
!endif

$(TCLSH): $(TCLSHOBJS) $(TCLSTUBLIB) $(TCLIMPLIB)
    $(CONEXECMD) -stack:2300000 $**
    $(_VC_MANIFEST_EMBED_EXE)

The _VC_MANIFEST_EMBED_* macros are described in a later section.

Note that most extensions do not need to explicitly use these macros as the default targets described later already do what is required for extensions. The macros are needed either when building application executables (as opposed to extension libraries) or multiple libraries from one makefile (since the default targets will only build the project library).

For situations where additional control is required, the following lower level macros may be used.

Macro Description
link32 Linker application name
dlllflags Switches to pass to the linker to build a shared library extension.
conlflags Switches to pass to the linker to build a console application.
guilflags Switches to pass to the linker to build a gui application.
baselibs Windows and compiler libaries
tcllibs Tcl libraries (only for extensions)
tklibs Tk libraries (only for Tk extensions)

Windows resource files

Windows programs and DLL's are expected to have a version resource embedded in them though this is not mandatory. To free the extension author from having to write a resource definition file, the nmake build system will automatically generate one and embed it into the built DLL extension.

If the standard template is not adequate for whatever reason, a custom resource definition file may be specified by defining the RCFILE macro before including rules-ext.vc in the extension makefile.

RCFILE = custom.rc

By default, the nmake system will look in the win and win\rc directories for this file. If located elsewhere, define the RCDIR as the path to the directory where the file is located. Again, this must be defined before inclusion of rules-ext.vc. For example,

...
RCDIR = $(ROOT)\rc
!include "rules-ext.vc"
...

(ROOT is automatically defined as the root of the extension source tree.)

All of the above will make use of implicit rules defined in rules.vc. If more control is desired, for example to pass additional flags to the resource compiler, an explicit rule needs to be added to the extension makefile after including rules-ext.vc. For example,

$(TMP_DIR)\custom.res: $(RCDIR)\custom.rc
    $(RESCMD) /DCUSTOMFLAG=1 $<

Note the use of TMP_DIR and RCDIR in the dependency. The RESCMD macro defines the command and associated standard flags to compile a resource definition file.

Generating pkgIndex.tcl

The standard default install target expects a pkgIndex.tcl file to have been generated and placed in the $(OUT_DIR) output directory. It then copies this into the extension's installation directory as part of the install. To generate this file, the top level target that builds the project has a dependency on the pkgindex target which is responsible for generating this file.

For a pure binary extension (without any additional Tcl script files), the following line added after including targets.vc

pkgindex: default-pkgindex

will generate this file with no additional work required on the author's part.

Alternately, if there is a pkgIndex.tcl.in file in the extension root directory that is used with the TEA build environment, the default-pkgindex-tea target can be used instead.

pkgindex: default-pkgindex-tea

In this case, the pkgIndex.tcl is generated from pkgIndex.tcl.in by replacing all occurences of @PACKAGE_VERSION@, @PACKAGE_NAME@ and @PKG_LIB_FILE@ with the values of macros $(DOTVERSION), $(PROJECT) and $(PRJLIBNAME) respectively.

For all other cases, the extension author needs to write their own target commands for pkgindex that results in a pkgIndex.tcl file being placed in the $(OUT_DIR) directory.

Targets

The default target

If unspecified on the command line, the default target is assumed to be $(PROJECT). To change this, define the macro DEFAULT_BUILD_TARGET before including rules-ext.vc. For example,

DEFAULT_BUILD_TARGET = release

Note you have to define the target yourself unless it is predefined or standard.

Predefined targets

Since most extensions build commands in a similar fashion, the rules.vc file provides some predefined targets that encapsulate the requisite commands. These are shown in the table below.

Target Description
default-test Runs the extension test suite assuming the script all.tcl in $(TESTDIR)
default-pkgindex Generates a pkgIndex.tcl file for a pure binary extension in the output directory.
default-pkgindex-tea Generates a pkgIndex.tcl file in the output directory using a TEA pkgIndex.tcl.in file as a template.
default-install Combines default-install-binaries and default-install-libraries.
default-install-binaries Copies the generated binary extension to the installation directory.
default-install-libraries Copies all *.tcl files from $(LIBDIR), if it exists, to the installation directory.
default-install-stubs Copies the extension stub library to the installation directory.
default-clean Cleans up $(TMP_DIR) but not $(OUT_DIR)
default-hose Cleans up $(OUT_DIR) which normally also includes $(TMP_DIR)
default-distclean Maps to default-hose plus cleans up nmakehlp helpers.
default-setup Does common build initialization like creation of output directories.
default-install-docs-n Copies all *.n files from $(DOCDIR) to $(DOC_INSTALL_DIR).
default-install-docs-html Copies all *.html and *.css files from $(DOCDIR) to $(DOC_INSTALL_DIR).
default-install-demos Copies the $(DEMODIR) directory to $(DEMO_INSTALL_DIR).

These pre-defined targets are used as building blocks by the standard targets described in the next section as well as potentially by additional custom targets defined by a makefile.

Standard targets

Inclusion of the file targets.vc defines "standard" targets commonly defined in makefiles and shown in the table below. These generally map to the predefined targets described earlier but can be extended as discussed in the next section.

Target Description
$(PROJECT) The name of the project builds the binary extension.
install Maps to default-install.
clean Maps to default-clean.
hose Maps to default-hose.
realclean Same as hose.
distclean Maps to default-distclean.
test Maps to default-test.
shell Starts a Tcl shell with the built extension's path added to auto_path.

Extending targets

In some cases, the standard targets "almost" suffice but are missing some additional actions. In such cases, instead of not including the targets.vc file and defining your standard targets, you can extend the standard targets by adding dependencies. For example, the default rules do not generate and install documentation as Tcl extensions do not have a common format and structure for the same. In simple cases like this, the appropriate target can be extended with additional commands:

install: install-docs
install-docs:
    $(CPY) $(DOCDIR)\*.html "$(DOC_INSTALL_DIR)"

Note CPY, DOCDIR and DOC_INSTALL_DIR are all predefined or computed macros within rules.vc.

Ignoring standard targets

In the extreme case, the standard targets can be completely ignored by the extension makefile by not including targets.vc. Note the built-in targets can still be accessed as default-install etc. as these are defined within rules.vc, not targets.vc. This is useful if for example, you wish to override the targets for building but use the default cleanup. In such a case, instead of including targets.vc the following targets can be defined in the extension makefile.

$(PRJLIB):
    ...Your custom build command to build the extension...
install: default-install
clean: default-clean
hose: default-hose

As an aside, when defining your own commands for building targets, you are strongly encouraged to make use of the predefined compiler and linker flags such as $(pkgcflags) etc. described elsewhere.

Stubs

There are two separate contexts in terms of discussing stubs when it comes to Tcl extensions.

  • The first is whether the extension should link with the Tcl (and potentially Tk) stubs library or directly with the Tcl DLL import library.

  • The second only applies to the case where the extension also supplies its own stubs interface, e.g. as is done by the tdom extension.

Linking with Tcl and Tk stubs libraries

By default, extensions are linked with the Tcl and Tk stubs libraries. In effect, this means /DUSE_TCL_STUBS=1, /DUSE_TCLOO_STUBS=1 and /DUSE_TK_STUBS=1 are passed to the compiler.

If it is desired to directly link with the Tcl and Tk DLL import libraries, pass nostubs as one of the values of the OPTS macro on the command line.

Generating extension stubs libraries

If an extension has its own stubs library for exported functions, define the macro PRJ_STUBSOBJS as the name of the source file containing the stub definitions. The default targets will then create a corresponding stubs library for the extension with the name $(PROJECT)$(VERSION).lib and place it in the $(OUT_DIR) directory. See the tdom extension for an example.

Version macros

By default, the nmake build determines the version of the extension by looking for a TEA configure.in or configure.ac file in extension's source root directory $(ROOT). The extension's version is extracted from the AC_INIT macro in that file. This allows the version to be maintained in a single place for the two build environments.

If the version cannot be determined as above or if the extension needs to have a different version on Windows for whatever reason, the macro DOTVERSION must be defined before including rules-ext.vc in the extension's makefile. This must be the major and minor version of the extension in dotted form. For example,

...
DOTVERSION = 1.2
!include "rules-ext.vc"
...

After including rules-ext.vc the DOTVERSION and VERSION macros are guaranteed to be defined. The latter is the DOTVERSION value with all periods removed.

If required, extension makefiles can access the Tcl and Tk versions against which the extension is being built through the TCL_VERSION, TCL_MAJOR_VERSION, TCL_MINOR_VERSION, TCL_PATCH_LEVEL, TK_VERSION, TK_MAJOR_VERSION, TK_MINOR_VERSION and TK_PATCH_LEVEL macros.

Installation

The rules.vc file defines the macros in the table below containing paths to installation directories.

Macro Directory
PRJ_INSTALL_DIR Main directory where package is to be installed.
BIN_INSTALL_DIR Directory where package binaries are to be installed.
SCRIPT_INSTALL_DIR Directory where package Tcl scripts are to be installed.
LIB_INSTALL_DIR Directory where package static libraries are to be installed.
INCLUDE_INSTALL_DIR Directory where package include files are to be installed.
DOC_INSTALL_DIR Directory where package documentation is to be installed.
DEMO_INSTALL_DIR Directory where demo scripts are to be installed.

As described in a previous section, the default-install-* set of built-in targets make use of these macros to install various components.

Manifests

Newer Visual C++ compilers support manifest resources in programs and DLLs. By default the built-in targets embed a manifest. If this is desired, define the PRJ_MANIFEST macro to contain the path to the manifest file. The built-in targets will then use this manifest file as a template, replace the token @MACHINE@ with the appropriate architecture, X86 or AMD64 and embed it into the generated DLL.

To explicitly embed a manifest in custom rule, you can use one of the macros _VC_MANIFEST_EMBED_DLL or _VC_MANIFEST_EMBED_EXE depending on whether the rule is applied to a DLL or a program.

Documentation and demos

Because documentation generation is not standardised across Tcl and extensions, the nmake system does not define any built-in rules related to demo scripts or generation of documentation.

Only some basic installation support is included through the default-install-demos, default-install-docs-n and default-install-docs-html targets described earlier.

Utility macros

The rules.vc file defines some utility macros to use as commands that operate on the file system.

Macro Description
RMDIR Deletes a directory and all its content.
MKDIR Creates a directory if it does not exist.
COPY Copies a file.
CPY Copies one or more files.
COPYDIR Recursively copies a directory.

Sanity checking build configuration

In general, it is recommended that extensions be built with the same configuration options as the Tcl it is being built against. For example, if the Tcl is linked against the debug C runtime, it is recommended that the extension also be linked against the debug runtime. For this purpose, a file tcl.nmake containing the relevant Tcl build options is created in the lib\nmake directory under the Tcl install. When compiling an extension, these options are checked against the build options specified for the extensions and warnings printed on incompatibilities.

These checks can be disabled by including noconfigcheck as one of the values passed through the OPTS macro on the command line.

Reference implementation

The vc-reform branch in the core.tcl.tk repositories contains work-in-progress. In addition to Tcl and Tk, the sampleextension, thread, tclvfs, tcludp, tdom, mpexpr and tktreectrl have been converted.

Examples

The aforementioned extensions may be used as examples of makefiles of varying degrees of complexity.

  • The sampleextension extension illustrates a basic, minimal makefile.

  • The tclvfs extension is an example that needs Tcl sources, makes use of the TEA pkgIndex.tcl.in file and installs man pages.

  • The tdom extension illustrates handling additional source directories, optional extra libraries, building the extension's own stubs file and generating a custom pkgIndex.tcl.

  • The tktreectrl extension is an example of a Tk extension.

For examples of defining custom targets and direct use of the build macros, see the makefiles for Tcl and Tk themselves.

Incompatible changes

For Visual Studio 2015 and later compilers, the compiler version number encoded in the default output directories now includes the entire version number, for example 1900 as opposed to the older mapped major version like 12. The reason for this is that Microsoft no longer necessarily changes the major version between releases but we still want to distinguish output directories from different compiler versions.

Copyright

This document has been placed in the public domain.

History
This page was generated in about 0.008s by Fossil 2.7 [9aa9ba8bd2] 2018-09-22 17:40:11