Author:         Colin Macleod <colin_g_macleod@yahoo.com>
State:          Draft
Type:           Project
Vote:           Pending
Tcl-Version:    9.1
Created:	23-Jun-2023

Abstract

This TIP proposes adding a new command calc which will allow numeric and boolean calculations to be written in a more compact form than with expr. The calc command will evaluate expressions compatibly with expr but in unbraced, presubstituted form.

It also proposes to make a standard predefined alias of = to calc which will permit numeric computations to be written within other commands in a more compact and natural form than at present, with no modification to Tcl's parsing rules.

Rationale

Most newcomers to Tcl, and some oldies, find expr awkward. The requirement to brace expressions for safety and performance leads to e.g. a canvas command with computed coordinates looking like:

.canvas addtag enclosed [expr {$x - 20}] [expr {$x + 20}] \
        [expr {$y - 20}] [expr {$y + 20}]

The wiki page https://wiki.tcl-lang.org/page/expr+shorthand+for+Tcl9 records many suggestions for a more compact syntax, one of which has also been proposed in TIP 672. This shows that the issue has been a concern for many years. However all of these proposals involve changing the basic Tcl parsing rules (the dodekalogue), which has a major impact in terms of extra complexity and backward compatibility. This TIP aims to allow such inline expressions to be as concise as possible without changing Tcl's parsing rules. The effect will be to allow the canvas command above to be written as:

.canvas addtag enclosed [= $x - 20] [= $x + 20] [= $y - 20] [= $y + 20]

Note that the values of variables (x and y in the example above) will have been substituted into the expression before calc is invoked, calc itself does no further substitutions of any kind.

There are some downsides to this method:

• String and list arguments and operations cannot be supported, as arbitrary strings could have values which cause them to be misinterpreted as operators, parentheses or numbers - since the quoting around them would be stripped off before calc saw them.

• Lazy evaluation of &&, || and ?: would not be possible.

• The tokens of the expression must all be passed as separate arguments, e.g. [= $x - 20] not [= $x-20]. This is necessary to avoid variable substitutions introducing new syntax elements, and also to avoid shimmering of numerical values.

But many uses of expr are for simple numeric calculations where these restrictions do not matter, but brevity is desirable. The standard expr would still be available for use in the more demanding cases.

An alternative which already exists for inline calculations is to use operations from the mathop namespace in prefix form. However this is rather obscure to people who are not Tcl experts, and becomes awkward if several different operators need to be combined.

Specification

The calc command will have syntax:

• calc arg ?arg arg ...?

It will evaluate expressions in a way which is compatible with expr with the following differences:

• Only numerical and boolean values and operations are supported.

• No substitutions (variable, command, backslash) will be performed by the calc command. Any variable or other substitutions which are desired should be done by the usual Tcl means before the calc command is invoked, therefore the arguments to calc should not be braced.

• Each syntactic token of the expression must be passed as a separate argument to calc, i.e. each numeric or boolean value, each operator or parenthesis must be separated by spaces.

The following alias will be predefined:

interp alias {} = {} calc

Defining this as an alias will allow any existing code which defines an "=" command to continue working. However new code can use [= <expr>] as a compact way to make a calculation.

Options

• Concerns were raised on tcl-core that not being able to support lazy evaluation of &&, || and ?: in calc could cause confusion. If this is felt to be a problem, these operators could be excluded from calc altogether, just as they are not included in the tcl::mathop namespace.

Discussion

The first draft of this TIP proposed implementing this functionality as an option on the expr command so that the arguments would be concatenated and then tokenised as expr does. Peter Da Silva pointed out that this would allow arguments intended as single values to introduce new syntactic elements, potentially changing the entire meaning of an expression. E.g.

set b 3/0
...
calc $a - $b
=> divide by zero!

To avoid this I decided to require the arguments to be separate tokens, so that no substitution of values can introduce new syntactic elements. At the time I thought this might also enable supporting string and list values, but later realised that it's not that simple. E.g. if the parser sees "(" it has no way of telling whether this is the start of a parenthesised subexpression or just a string value that happens to contain "(".

So it then became clear that this functionality was sufficiently different from expr to make it a separate command. Also when I looked into the expr parsing code I realised that a separate implementation would be more practical.

At one stage I considered making invoking calc with a single argument a special case, just returning that argument with no parsing or processing. However this would prevent detecting what could be a common error case - passing the expression without spaces as can be done with expr - so I concluded it would be unwise.

Examples

Setting a variable:

    set bright [= $red * 0.3 + $green * 0.59 + $blue * 0.11]
    set x [= $radius * cos( $angle )]

Use with an image command:

    my_img put $shade -to [= $left + $i] $top [= $left + $i + 1] $bottom

Implementation

I have written a prototype of this functionality in Tcl, the code is at http://www.cmacleod.plus.com/tcl/calc.tcl. This code uses a simple "Pratt" parser and generates bytecode which is then run by tcl::unsupported::assemble . For the real implementation my intention would be to translate the same code into C.

I think byte-compilation of calc should be possible and worthwhile. For the normal case where all operators and parentheses are written as literals and only numeric or boolean values will be substituted at run-time, it should be possible to do the parsing at compile-time and generate reusable bytecode. However there would need to be a run-time check that the substituted values are actually numeric or boolean. E.g. the command

calc $a - 2

would be compiled assuming that $a has a numeric value and therefore the "-" is a infix subtraction. But if $a should have the value "-" we need to reparse this as two unary minus operators. Such cases can be expected to be rare, but do need to be handled. So the compiled bytecode needs to check that all substituted values are numeric or boolean and fall back to the uncompiled implementation if this does not hold. I'm not sure how to write this yet but think it should be possible. Perhaps it would be enough to just run the bytecode and if it returns an error then somehow fall back to the uncompiled implementation?

Copyright

This document has been placed in the public domain.