Disadvantage and basic factor of to

[ruv]

Sometimes value-flavored variables are very convenient.

One problem with them is that they are bad in meta programming, because there is no standard method to perform the action of to when you have a name token.

In typical usage the word to references a word name as:

0 value foo  \ create the word "foo" (that is a value-flavored variable)
1 to foo \ assign 1 to the word "foo"

But if we have the name token nt for foo:

s" foo" find-name ( nt )

there is no standard way to assign 1 to the word foo.

For defer-flavored variables we have such a way — the words defer@ and defer! that apply to xt.

---

What a word "X" (or maybe several words) can you suggest so it can be used to assign a value to a value-flavored variable through its nt or xt?
(Update: also, "X" may apply to a qualified semantic token qt instead of nt; qt is returned by a recognizer).

1. The word "X" must hide implementation details and be implementable in any Forth system that provides value and to. An implementation for "X" does not matter.
2. The word "X" may depend on state, if you like.
3. Could you please provide an implementation for the word to using the word "X" (or several words)?

Note that to applies to local variables, so "X" should apply to local variables as well.

[MitchBradley]

Most of my Forth implementations have (TO) ( value xt -- )

Implementations vary, from

: (to)  ( n xt -- )
   dup >body  swap cf@         ( n 'body cf )
   dup (value)      =  if  drop >user !      exit  then
   dup (defer)      =  if  drop >user token! exit  then
   dup (user)       =  if  drop >user !      exit  then
   dup (vocabulary) =  if  drop >user token! exit  then
\   drop !
   drop body> to-hook
;

to

: (to)  ( value xt -- )  +level  compile to  (compile) -level  ;

+level enters temporary compilation state, while -level exits that state and executes the code fragment that was just compiled. They are primarily used for interpreted control structures.

The (xxx) naming convention was in vogue a long time ago, but AFAIK it is no longer common.

[ruv]

And how do you define to using (to)? It's not clear how this can work for local variables.

[ruv]

@MitchBradley wrote:

Here's the definition for the first version. For the second version, (TO) is defined in terms of TO

: to  ( "name" [ val ] -- )	\ val is present only in interpret state
   state @  if   postpone ['] postpone (to)  else  ' (to)  then
; immediate

I doubt that it works with locals. I only added locals late in the game, and rarely use them. Making everything work with them, and implementing the code elegantly, never became a priority.

Here is some ugly overload code that makes TO work with locals

variable to?  to? off

: do-local  ( local# -- )
   postpone literal
   to? @  if  postpone set-local  else  postpone get-local  then
   to? off
;

\ This is the word defined in Basis
: (local)  ( adr len -- )
   dup  if
      #ins @  ['] do-local local-name
   else
      2drop
      #ins @  postpone literal   postpone allocate-locals
   then
;

warning @ warning off
\ Redefine TO to handle the local variable case also
: to  ( "name" [ val ] -- )	\ val is present only in interpret state
   >in @ >r  '  r> >in !             ( xt )	\ Peek at next name
   ['] do-local-name token@  =  if          \ Do the local variable thing
      to? on                         ( )
   else						\ Call the old version
      postpone to
   then
; immediate
warning !

So, for both value children and local variables, you operate on xt.

For the former you operate in run time (or execution), for the latter — in compilation time (and do-local is executed in that time).

Below I'm trying to factor out the common word (to), which operates in both cases:

: compilation ( -- flag ) state @ 0<> ;

\ Assumption: xt-local performs the compilation semantics for a local variable.

: (to) ( comp: false ; x xt-value --  |  comp: true ; xt-local|xt-value -- )
  dup ['] do-local-name token@ = >r \ whether it's xt for a local variable
  compilation if  ( xt-local|xt-value )
    r> if ( xt-local ) \ a local variable
      to? on  execute  exit
    then ( xt-value ) \ a common value-flavored variable
    postpone literal  postpone (to)  exit  \ compile the old version
  then  ( xt ) r> if -14 throw then  ( xt-value ) (to)  \ call the old version
;

\ The definition for `to` becomes very simple:
: to  ( comp: false ; x "name" --  |  comp: true ; "name" -- )
  '   (to)
; immediate

This seems like a possible option.

[forthy42]

IMHO, a word to! and corresponding +to! could do that job (>addr for the use of addr on the value/variable crossover varue). Of course, all the magic behind to and friends is very implementation-specific. In Gforth, you can define that similar to defer! with the corresponding magic numbers (0 for to, 1 for +to, and 2 for addr.

: to! ( x xt-value -- )
    [ 0 ] (to) ;
opt: ?fold1 0 swap (to), ;

Systems where to and +to just sets a one-shot variable that is evaluated and cleared when the following value is executed or compiled are easier, you can define

: to! postpone to execute ;

and the optimization part would do postpone to compile, after the system-specific equivalent to opt: ?fold1.

[forthy42]

This works for locals only if the optimization is successful, i.e. the xt of the local is provided as foldable constant. The interpretative part for locals always throws an exception -14, it must go through the optimization path.

The definitions above therefore won't work with locals on gforth-itc, which turns optimization off; the parsing words circumvent this problem by turning optimization on temporarily.

[ruv]

This works for locals only if the optimization is successful, i.e. the xt of the local is provided as foldable constant.

Then, if I understand that right, the following definition for to should work for local variables as well.

: to  ( comp: false ; x "name" --  |  comp: true ; "name" -- )
  '   state @ if postpone literal postpone to! exit then  to!
; immediate

Folding of constants is cool. However, this approach is not implementable in all Forth systems, and mentioned gforth-itc is such an example.

[ruv]

@forthy42 wrote:

a word to! and corresponding +to! could do that job (>addr for the use of addr on the value/variable crossover varue).

Given the naming schema of defer! and defer@ (which are not named like is! and is@), perhaps better names would be:

• value! ( i*x xt-value -- )
• +value! ( i*x xt-value -- ) (update: or better value+!)

The name >addr ( xt-value -- a-addr ) follows the convention of >body ( xt -- a-addr ). There are some other naming options that do not contain the «>» character and are more specific than just addr:

• addr-of-value ( xt-value -- a-addr )
• addr-of-varue ( xt-value -- a-addr )

[ruv]

• +value! ( i*x xt-value -- )

Regarding a basic factor of the word +to.

It is probably better if this word name follows the naming convention of the following well known words:

• — >r — f>r
• — +! — f+!
  • On GitHub, the name f+! appears in Forth programs, but the name +f! does not appear.

Then, the basic factor of the word +to should be named as value+!.

The stack diagram (for standard cases) is:

• value+! ( S: n|u|d|ud xt-value -- ; | F: r -- ; S: xt-value -- )
  And the data type of the second parameter is determined by the specific xt-value subtype.

A more narrow stack diagram is:

• value+! ( n|u xt-value-single-cell -- ; | d|ud xt-value-double-cell -- ; | F: r -- ; S: xt-value-float -- ; )
  Where the following data type subtyping relationships are satisfied:
  • xt-value-single-cell ⇒ xt-value
  • xt-value-double-cell ⇒ xt-value
  • xt-value-float ⇒ xt-value
  • xt-value ⇒ xt

[forthy42]

Yes, the names value! and value+! make sense.

[ruv]

In most Forth systems, if the system provides the word «value!», the phrase «['] foo value!» is not compiled as efficient as the phrase «to foo» (a similar case for «compile,» is mentioned in Rationale / A.6.2.0945). Moreover, «value!» cannot be used for local variables (for which «to» applies).

To make «to» implementable (in a standard program), the following words might be introduced along with value! ( x|xd|r xt -- ):

• compile-store-local ( lvt -- ), where lvt is a symbol for a local variable token;
  • this word appends the "TO name" run-time semantics associated with lvt to the current definition;
• recognize-local ( sd.name -- lvt tt-local | 0 );
  • this word tries to recognizes sd.name as a name of a local variable;

And adding of the following word makes «to» implementable efficiently:

• compile-store-value ( xt -- );
  • this word appends the "TO name" run-time semantics associated with xt to the current definition;

To make «action-of» and «is» implementable efficiently, the following words might be introduced:

• compile-store-defer ( xt -- );
• compile-fetch-defer ( xt -- );

---

Bottom line: the lack of standardized basic factors is a significant drawback of to, is and action-of.

[ruv]

One more thought.

Forth natively provides low-level compilation methods: «compile,», «literal», «2literal», «fliteral», «sliteral». These methods do perform neither text translation nor recognizing of lexemes, but operate on basic data objects — execution tokens, numbers, strings.

The standard does not provide methods to compile control flow constructs and rstack-related operations, but they can be constructed from high-level words (e.g. «if») using «postpone» (e.g. «[: postpone if ;]» gives xt that do compilation).

The only entities that fall out entirely are value-flavored variables (including local variables).

For example, using standard means we can redefine some words so that a colon definition is compiled into an intermediate representation, then optimized in high-level (using transformation, inlining, substitution result of pure functions, etc), and then the actual definition is compiled. The only elements that cannot be efficiently supported are assignments of value-flavored variables.

Standardizing low-level methods to compile assignment of value-flavored variables closes this gap.