Author:         Sean Woods <yoda@etoyoc.com>
State:          Draft
Type:           Project
Vote:           Pending
Created:        23-Oct-2017
Post-History:
Keywords:       Tcl,procedure,argument handling
Tcl-Version:    8.7

Abstract

This TIP proposes an enhancement of the Tcl language to support named parameters when defining a procedure or OO method.

Rationale

Adding a new argument to commands and methods is fairly simple in Tcl. Finding all of the references in your code where you call that command (or method) and padding in extra arguments, however, is a painful process. A pain that is magnified exponentially with the size of your project. While Tcl does provide the args facility for optional arguments beyond the positional arguments, that facility does not populate local variables, nor provide any meaning or context beyond Thar be dragons. Well, dragons and a list called args that the called procedure needs to sort out by itself.

In the Tk world we mitigate this complexity by having as few positional arguments as possible, and instead providing a rich set of optional configuration flags. The flipside is that those options need to be registered ahead of time in a massive option database. Also, those option/value pairs go directly into C data structures, they at no time feed a script with local variables.

Specification

Currently if the last parameter named args is given a default value, that information is essentially ignored by the workings inside of tclProc.c. This spec calls for storing a dict stored where a default would go for the args parameter. Each key in the dict will be the name of a local variable that the parameter will populate. The values in the dict wil be a key/value set of configuration option for the named parameter system.

Developers can specify if a named parameter has a default if not given. They can also specify if the parameter is required. A non-mandatory parameter will not be mapped to a local variable, but it is understood that developers may want to advertise their presence for documentation purposes.

	proc foo {{{args {
	   bar {default: baz}
	}}}} {
	   puts $bar
	}
	% foo
	> baz
	% foo bar bing
	> bing

The primary use case for this tip are new functions that take only named parameters. This TIP also calls for the creation of a new command, provisionally named procx. procx will accept the same number of arguments as proc, but the format for the argument spec will be a dict instead of a list. For TclOO methods, we will also create an methodx as a shortcut to:

	oo::define myclass method namemethod {{{args {
	   bar {default: baz}
	}}}} { ... }

The workings of procx and methodx are essentially:

	proc procx {name argspec body} { proc $name [list [list args $argspec]] $body }

Named parameters are designed to be deliverable in any order. Named parameters can also be omitted. As such it is necessary to provide more introspection than the current incarnation of proc provides. If the procedure requires more than the default behavior or populating local variables with the argumemnt given, it can introspect with $args with [dict exists] to detect if an argument was given at all. The argument spec that was given by the developer will also be available as the local variable argspec.

	% procx p {a {default: {}}} { return $args }
	% p
	a {}
	% p foo bar
	a {} foo bar

Given these rules, we can get into non-intuitive failures. For instance, what would happen if a default value was not provided for an argument, and the command call does not provide one.

	% procx q {a {default: {}} b {}} {
	  # Proc assumes b will be given
	  set result [list $a $b]
	  foreach {f v} $args {
	    if {$item ni [dict keys ${argspec}]} {
	       lappend result $item [dict get $args $item]
	    }
	  }
	}
	% q
	error: No such variable b ; # Stack trace points to first use if $b
	% q b foo bar baz
	a {} b foo bar baz

It would be helpful to have that field (b) treated as a mandatory argument, and have the error thrown in the procedure's argument parser, not in the procedure itself. After some play testing with the rules I have worked out what properties will be tracked for each named parameter, and how those properties are calculated if not specified directly in the options.

Configuration Options for Named Parameters

The following general rules apply to all values given in the configuration dict for a named parameter.

If a value for any option is given as an empty string, it is inferred to be null.
A variable will be created with the same name as the named parameter.
A field that is given, but not otherwise reserved by the spec, is ignored by the parser but made available for introspection via the argspec dictionary.
If the same option is given multiple times, the last value is the one that will be used. (Similar to [dict merge]).

mandatory:

Type: boolean Default: true

Set to false if default given. If true, throw an error if the named parameter is not given.

default:

Type: value Default: null

When non-null, if a named parameter is missing from the call assume the value specified.

Example:

	% procx u {
	  a { default: {A useful value} }
	} {
	  puts $a
	}
	% u
	> A useful value
	% u a {Less usefull}
	> Less usefull

aliases:

Type: list Default: null

A list of alternative names for this parameter. To resolve any potentials conflicts, the following rules apply to aliases:

If a parameter with the canonical name is given, alternatives will be ignored.
If multiple non-canonical parameters are given, the last value given will be the value for the canonical field.

Example:

	% procx u {
	  a {
	    default: {A useful value}
	    aliases: {alpha A}
	  }
	} {
	  puts [list $a $args]
	}
	% u
	> {A useful value} {}
	% u a {Less usefull}
	> {Less usefull} {a {Less usefull}}
	% u alpha {Less usefull}
	> {Less usefull} {a {Less usefull} alpha {Less usefull}}
	% u a {Don't be pedantic} alpha {Less usefull}
	> {Don't be pedanditic} {a {Don't be pedantic} alpha {Less usefull}}

type:

Type: string Default: null Options: null, boolean, integer, wide, entier, double

To support tip#480. For now strictly advisory. In the future this will check that the incoming values are the specified type.

Implementation

This TIP will be rolled out in 3 stages.

Stage 1 - Pure Tcl (finished)

Stage 1 is a pure-tcl implementation to allow the community to try out the rules and see if this new concept is a good fit for Tcl. The implementation will use a macro-like template to place the implementation rules as a pre-amble to the body given to procx or methodx. This implementation will affect line numbers in traces, but should provide the new features without introducing a major impact on performance.

This implementation has been posted to the http://wiki.tcl.tk/49071 and will be kept up to date.

State 2 - C Extension (finished)

Stage 2 will be a standalone TEA extension compatible with Tcl 8.6. In this implementation a C API call will create a command argsx which will take as input the parameter specs for the proc and the value of the args variable. The argsx command will then provide the local variables for this tip's rules prior to passing control to the user's body.

To build the extension:

  mkdir tip479
  cd tip479
  fossil clone http://fossil.etoyoc.com/fossil/tip479 tip479.fossil
  fossil open tip479.fossil
  tclsh make.tcl all

State 3 - Core patch

Stage 3 will introduce named parameters as a feature of the core. Modifications have been checked into the Tcl core as the tip479 branch.

At present if the last positional argument is named args a default value for that field, while recorded, is ignored. We will utilize that space to overload what arguments the procedure will accept beyond the positional arguments.

Within tclProc.c are modifications to read that argument spec (if given).

TclCreateProc

In TclCreateProc, if an named argument spec was registered for the function, additional localPtr entries are registered beyond args to hold the expected values added by the spec.

ProcWrongNumArgs

In ProcWrongNumArgs, the contents of the argument spec are reported instead of ?args ? if an argument spec was registered for that proc.

InitResolvedLocals

In InitResolvedLocals, we modify the loop to also resolve variables in the compiledLocals beyond the numArgs.

InitArgsAndLocals

During standard argument processing, the function notes if the procedure has a named argument spec registered. If a spec was present, after it performs its normal population of local variable matching the parameters for the procedure, the function calls ProcEatArgs which pairs key/value pairs beyond the last positional parameter with registered name parameters (and their associated local variable.

init.tcl

Two macros for procx and methodx are placed in the init.tcl file to ensure those facilities are present at runtime. The community is presently in a debate as to whether seperate commands are the best approach, or modifying proc and oo::define::method to accept flags is better.

info argspec

In the info namespace, add an argspec command to return the named argument spec for a command

Performance

Included in the file distribution is a test script which produces performance metrics. The most up to date metrics are posted to:

http://fossil.etoyoc.com/fossil/tip479/doc/trunk/performance.txt

Long story short, for the C implementation, processing arguments using the argsx mechanic is faster than performing [dict with args {}]. An equivilent proc with positional arguments is still slightly faster.

The core implementation is faster than the C implementation, without introducing any slowdown for conventional procs.

The code to produces the benchmarks is included in the code repository:

http://fossil.etoyoc.com/fossil/tip479/doc/trunk/tests.tcl

Extras

While this tip addresses the design patterns of proc definitions.

Arguments for ensemble-like procs

The argsx command is available for use at any place within Tcl, not just at the top of a command. Consider:

proc myensemble {command args} {
  switch $command {
    smtp {
       # Dicts can be created on the fly
       argsx [dict create from {} to {} mtime [dict create default: [clock seconds]]] $args
       smtp::dispatch $from $do mtime $mtime
     }
     http {
       # Or accept a static string with whitespace
       argsx {
          from {}
          to {}
          url {default: http://mydomain.info}
       } $args
       http::post ${url}?from=${from}&to=${to}
     }
  }
}

Extra Metadata for argument handling

The standard only defines the behavior for several properties for parameters, but does not prohibit storing additional properties. Consider:

procx prettyproc {
  color {
    comment: {Color of the button}
    type: color
    default: green
  }
  flavor {
    comment: {What flavor candy emerges when pressed.}
    default: random
    options: {vanilla caramel chocolate strawberry}
  }
} {
   set flavors  [dict get [info argspec prettyproc] flavor options:]
   if {$flavor eq "random"} {
     set flavor [lrandom $flavors]
   } else {
     if {$flavor ni $flavors} {
        error "Invalid flavor $flavor. Valid: random OR $flavors"
     }
  }
}

Spec Revisions

2017-11-03 - Renamed @proc to procx, @args to argsx and @method to methodx. Removed the internal variable @spec from bodies. Added

2017-10-28 - Removed the variable option. The core implementation really needs a canonical name for the variable in the index. If a user really wants a different name in the arguments, we have the aliases facility. Also clarified that this tip will be modifying how a default for the args parameter is interpreter and that procxs and methodx are just convenience wrappers.

TIP 479: Add Named Procedures as a New Command in Tcl (procx)