Author:         Donal K. Fellows <donal.k.fellows@manchester.ac.uk>
State:          Draft
Type:           Project
Vote:           Pending
Created:        10-Feb-2018
Post-History:
Keywords:       Tcl, object orientation, visibility
Tcl-Version:    8.7

Abstract

This TIP proposes a mechanism for (somewhat) private variables in TclOO.

Rationale

One of the principles of object oriented programming is that classes should be isolated from each other. This particularly includes the isolation of a superclass (which might be in one package) from its subclasses (in other packages) other than for its published API. The TclOO object system in Tcl 8.6 does not provide a way for user code to get such isolation: all classes that are collaborating in a hierarchy to implement a particular instance object see the same set of methods and variables. In the case of methods, it is because the classes end up placing their methods into the same general resolution scope (formed effectively from the union of all their methods), and in the case of variables, it is because all variables are actually just normal Tcl variables in the instance object's namespace.

A mechanism is desired to increase the isolation of classes from each other, even where they are in a superclass-subclass relationship. Note that any such mechanism does not need to be totally proof against prying (and introspection mechanisms are desirable) as TclOO deliberately does not provide security mechanisms; such mechanisms are the purpose of Tcl's interpreters. All that is desired is a way of ensuring that code does not inadvertently trample other code, allowing a superclass to evolve its implementation without needing to take into account every detail of its subclasses.

Detailed Proposal: Methods

For methods, as they actually exist in storage that is totally controlled by TclOO (and which is not addressable other than via TclOO-created commands or introspectors) the main complexity is working out what lookups are possible in a particular case while ensuring that method lookups are fast. Determining what the context class (or instance) is easy enough, as that's present pretty much directly in the Tcl_ObjectContext, but rapidly determining whether the object that is invoked is able to be seen in that way is the non-trivial part. (As with most things in TclOO, appropriate caching is the key to accelerating this sort of thing with a reasonable memory overhead.)

A key objective is that if one invokes $otherobj ClassPrivateMethod from inside a method that is part of the class but a different instance or possibly even a different subclass of that class, then that method will be found. That makes this a useful mechanism that is not provided by the TclOO system in Tcl 8.6.

Method Invoke Semantics

When a method is invoked, a public (or unexported, if via the self command) method is always preferred. However, if the resolution of the method would result in no resolution of the method, private methods are also searched. The private methods that are searched are those of the declaration context (class or method) of the currently executing method only; subclasses and superclasses are not examined. Filters are still applied to the outside of the method call, but the actual private method has no need to ever call next or nextto; it is always the terminal method implementation in its method call chain.

A private method on a class (or object) may not have the same name as another method on that class; all methods of a class (or object) are part of a single space of names. When resolving a method call of a subclass of the class that has the private method, even if the method names match, the private method does not participate in the call chain; only an exact match from exactly the right context counts.

In particular:

oo::class create Top {
    method Foo {} {
        puts "This is Top::Foo for [self]"
    }
}
oo::class create Super {
    private method Foo {} {
        puts "This is Super::Foo for [self]"
    }
    method bar {other} {
        puts "This is Super::bar for [self]"
        my Foo
        [self] Foo
        $other Foo
    }
}
oo::class create Sub {
    superclass Super
    method Foo {} {
        puts "This is Sub::Foo for [self]"
        next
    }
    private method bar {other} {
        error "This should be unreachable"
    }
}
Sub create obj1
Sub create obj2
obj1 bar obj2

is expected to print out:

This is Super::bar for ::obj1
This is Sub::Foo for ::obj1
This is Top::Foo for ::obj1
This is Super::Foo for ::obj1
This is Super::Foo for ::obj2

Note that the call via my picks up the unexported method from the subclass, and that the call from outside does not find the method on Sub, and the call to next does not pick up the method on Super.

Private methods on object instances are similar, and are only visible from other instance methods on the same object. The author expects them to be rarely used.

Creation and Introspection

Private methods are created calling oo::define (and oo::objdefine, and via the constructor of oo::class) like this:

oo::define TheClass {
    private method foo {...} {
        ...
    }
    private forward bar  SomeOtherCommand
}

More formally, the private definition command sets a flag for the duration of its execution that causes any methods (of any standard type) defined in the current definition context to be marked as being private rather than using the usual public/unexported semantics. All remaining arguments are then concatenated as a proper Tcl list and the result evaluated in the current context. It has no effect on recursive calls, or on the export or unexport definition commands, or on any custom declaration types.

At the C API level, private methods can be directly created by setting the flags parameter (called isPublic in older versions of the API) to the constant TCL_OO_METHOD_PRIVATE. This lets custom types of methods be declared by their C code directly.

Introspection is done via appropriate options to info class methods and info object methods. In particular, they are never returned when the -all option is given, are found when -private option is given (unless the -all option is also given), and can be found when the new option -scope is given, depending on what scope is chosen from the options below:

-scope public

Reports the public (public) methods only.

-scope unexported

Reports the unexported (non-public, created via naming or unexport) methods only.

-scope private

Reports the truly private methods only.

The -scope option causes the -all and -private options to be ignored.

Detailed Proposal: Variables

Variables are more complex, as they necessarily can be named by other commands (e.g., so that they can be vwaited upon). This requires them to be named by compounding the user-supplied name (to reduce confusion) and a unique identifier that identifies the particular context of the name. Now, one of the trickier things about TclOO is that it does not provide fixed names for objects; the rename command is supported. Because of that, the fully-qualified name of an object or class, while unique, is not a stable identifier. The full name of the backing namespace is stably-named (as namespaces cannot be renamed), but only has a unique name when the fully-qualified name is used; single components of the name are not guaranteed to be unique as users can control what those namespace names actually are. However, there is also another unique identifier inside TclOO, the object creation ID, which is based on a simple interpreter-level counter that is incremented for each time an object is manufactured. (The creation ID is used to allow very fast comparison between cached Tcl_Obj internal representations, as well as being the seed for the automatically-created object names.) This creation ID has a number of advantages, but its key ones are that it is guaranteed to be a simple token (an integer!) and it is guaranteed to be unique within an interpreter.

In particular, once a variable has been identified as being private by declaration, it is visible to the methods of the class (or instance) that defined it to be private with its given name, but it exists with a different actual name in the namespace. That actual name (which is theoretically globally visible once discovered) consists of the creation ID, a separator (a string consisting of a space, a single colon, and another space) chosen to be unlikely in any user script while also being disjoint from the syntax of arrays, and the user-supplied portion of the name. String substitution using these global names is not recommended. The variables concerned are not created by declaration; that's up to an appropriate method to actually do (and the variable can be created as normal scalar, array or link). When a method runs in the relevant class, the class's variable resolver will map the private variable in with the user-expected name.

The variable and varname methods of oo::object will be aware of the mechanism, respecting what names are visible in the context that invokes them.

Creation and Introspection

A private variable mapping is created like this:

oo::define Foo {
    private variables x y
    method foo {} {
        set x 1
        set y(z) 2
    }
}

That is, the private declaration command described above (for methods) also impacts the variables slot declaration in oo::define and oo::objdefine, causing it to introduce this more complex mapping pattern. Note that the variable resolver already is class-scoped.

In the above example, a subclass of Foo that refers to variables x and y will not see the variables defined by Foo, but will instead see either the current standard variable scheme or their own x and y. (If the creation ID of Foo is 123, then Foo's x and y will really be called 123 : x and 123 : y, respectively.)

Introspection is via adding an extra option to info class variables and info object variables, -private, which causes these commands to list the private variables. Without the option, the non-private (i.e., direct mapped) variables are listed.

A supporting introspector is also added, info object creationid, which returns the creation ID of any existing object. It also applies to classes. Again, note that creation IDs are always system-allocated and are never guaranteed to be unique between interpreters, either in multiple processes or in the same thread or process; they are only ever locally unique.

Implementation

Not yet done.

Copyright

This document has been placed in the public domain.