Author:		Ashok P. Nadkarni <apnmbx-public@yahoo.com>
State:		Final
Type:		Project
Vote:		Done
Created:	14-Nov-2022
Tcl-Version:	9.1
Tcl-Branch:	tip-649
Vote-Summary:	Accepted 9/0/0
Votes-For:		AK, AN, BG, DF, HO, MC, RA, RM, SL
Votes-Against:	none
Votes-Present:
Keywords:	list

Abstract

The list implementation has functionality that is available at the script level but not in the C API. This TIP proposes to expose the same as a C API as well. The benefits include

• convenience for extension and application writers that need to manipulate lists at the C level by eliminating the need to implement the functionality themselves

• significantly better performance as the currently defined list related C API is implicitly built around the 8.6 list internal representation and does not allow callers to benefit from implementation of more efficient higher level operations available in Tcl 9.

• prevention of shimmering from the more efficient list representations made possible by TIP 636 (for example, arithmetic series).

Specification

The following functions will be added along with corresponding entries in the stubs table.

int Tcl_ListObjRange(
    Tcl_Interp *interp,
    Tcl_Obj *srcListPtr,
    Tcl_Size first,
    Tcl_Size last,
    Tcl_Obj **newListPtrPtr);
Tcl_ListObjRepeat(
    Tcl_Interp *interp,
    Tcl_Size repeatCount,
    Tcl_Size objc,
    Tcl_Obj *const objv[],
    Tcl_Obj **newListPtrPtr)
int Tcl_ListObjReverse(
    Tcl_Interp *interp,
    Tcl_Obj *srcListPtr,
    Tcl_Obj **newListPtrPtr);

where

• interp is used for error messages and may be passed as NULL.

• srcListPtr points to the source list operand. This may be shared or unshared. The called function may store a reference to it internally so if srcListPtr is passed in with a 0 reference count, it should not then be freed with Tcl_DecrRefCount. Rather, use Tcl_BounceRefCount instead.

• newListPtrPtr points to the location to store the result Tcl_Obj. This is guaranteed to be distinct from the passed in srcListPtr even when the latter is unshared. The returned Tcl_Obj may or may not be shared so caller should check before modification and should not decrement its reference count without incrementing it first. On errors, *newListPtrPtr is set to NULL to detect unchecked errors but will only be documented as undefined.

Tcl_ListObjRange is the C equivalent of the lrange command and returns a list containing all elements in the source list at indices greater or equal to first and less than or equal to last. An empty list is returned in the case of first being greater than last.

Tcl_ListObjReverse returns a list containing all elements of the source list in reverse order.

Tcl_ListObjRepeat returns a list whose elements are the objc elements passed in objv repeated repeatCount number of times. repeatCount must be a non-negative integer.

Return values

All functions return TCL_OK on success and TCL_ERROR on failure.

Discussion

Reference counting conventions

For convenience as well as efficiency (avoid unnecessary duplication), the functions do not require srcListPtr to be unshared unlike the existing Tcl_ListObjReplace function.

Because many use cases will only be read-only (iteration etc.), the returned list is not guaranteed to be unshared, unlike Tcl_NewListObj. This is because some abstract list types will return internally referenced objects e.g. repeatedList.

The Tcl_Obj * returned in newListPtrPtr is guaranteed to be different than srcListPtr even when srcListPtr is an unshared Tcl_Obj. The intent is to simplify reference count management for the caller. We want the caller of the function that is operating on a list to be able to treat the passed in srcPtr and resultPtr independently when it comes to managing reference counts. Otherwise, it is very easy for the caller to mess up the reference counts of the two objects by not checking the result object is the same as the source object before decrementing reference counts for both, or incrementing and decrementing in the wrong order. To avoid this, we always return a new object. This has a small sacrifice in performance that is thought worthwhile to protect against errors in the caller.

Obsoleted tests

Two lrange tests, lrange-4.3 and 4.4 that check (in part) that unshared Tcl_Obj are reused have been modified to remove that check. With the faster implementation of lrange in the new internal representations of lists, "hacks" like

lrange $l[set l {}] ...

should no longer be necessary as there is no measurable different even for large lists.

This change stems from the design decision above to not reuse the Tcl_Obj passed in for the result even when it is unshared.

Implementation

Implementation is in the tip-649 branch. Test cases for C API are in the listTypes.test file.

Copyright

This document has been placed in the public domain.