Another bad idea: $&whatis, settor variables, and a new primitive

[jpco]

A natural question to ask when encountering es is: why doesn't variable assignment work via a hook function? Where is $&set (or $&assign, or whatever)?

Another question which has come up is "why is $&whatis a primitive? it shouldn't have to be, should it?"

The typical answer to both of these (in terms of language design, rather than implementation/performance considerations) is "lexical binding". There's no "native" way to define an es function which runs within its caller's lexical context, so the only way to get behavior that is "lexically aware" like you need for $&set and $&whatis is through C code (which might call out to es code in a special limited context, like with settor variables).

However, it's actually reasonably simple to implement something which does this. My idea here (of which I have an actually working implementation!) is the primitive $&keeplexicalbinding. It's used like this:

fn-echo-foo = $&keeplexicalbinding @ {
  echo $foo
}

foo = bar
let (foo = WOW) echo-foo

Without the $&keeplexicalbinding, echo-foo does the (objectively correct, lexical-binding-respecting) thing and echoes bar. With the $&keeplexicalbinding, however, echo-foo does the absolute wrong thing and refers to the lexically-bound foo -- echoing WOW.

This punches a major and disturbing hole through the lexical binding semantics of es. However, it also may enable some neat things, like making %whatis into a pure-es function! Something like:

fn-%whatis = $&keeplexicalbinding @ var rest {
  $&if {~ $(fn-^$cmd) ()} {
    if {~ $cmd /* || ~ $cmd ./* || ~ $cmd ../*} {
      result $cmd
    } {~ $fn-%pathsearch ()} {
      result ()
    } {
      %pathsearch $cmd
    }
  } {
    $&result $(fn-^$cmd)
  }
}

This function would be shared by the user calling whatis foo and the shell itself when deciding how to run anything that isn't a lambda, primitive, or other special form.

In the same vein as %whatis could be a general %set function. Something vaguely like

fn-%set = $&keeplexicalbinding @ var value {
  if {~ $(set-^$var) ()} {
    value = <={set-^$var $value}
  }
  if {!~ $var $noexport} {
    $&export $var  # or $&setenv, or whatever
  }
}

(This function, I think, wouldn't get called for lexical bindings... in which case maybe the $&keeplexicalbinding isn't necessary?) Then in theory another hook function could be used for $var, and that hook function could do the corresponding $&getenv as appropriate. That hook function, of course, would strictly need $&keeplexicalbinding in order to fetch lexically-bound variables.

Users could also extend %set to add new behaviors similar to noexport, like a readonly variable. (Would anybody find readonly useful? That seems like it's in the direction of a restricted shell mode. It could be fun to experiment with using the shell's flexibility to restrict users of the shell.)

Exposing the mechanism by which noexport works should also be sufficient (along with hacking %interactive-loop) to allow users to whole-hog switch between the rc style "login script/noexport/pass everything through the environment" and bash style "login script/export/interactive init script" as they wish. That seems potentially practically useful -- while I like the rc-style behavior, it seems extremely reasonable to me to want to avoid the mess it makes of the environment.

The mailing list can be trawled for other ideas which don't work due to hook functions losing the caller's lexical binding.

I'm not sure any of this is a good idea. But it is possible, and I don't know if the community has actually believed it was possible before.

[jpco]

I have been convinced over time that the above $&keeplexicalbinding primitive is a bad idea.

However, I do still think that %whatis and %set are pretty good ideas anyway.

%whatis could be done like in https://github.com/jpco/es-shell/tree/whatiser -- the shell internally resolves function definitions (so let-bound functions would still work), but anything else is passed to the %whatis function, which can convert the call to a %run <={%pathsearch blah} blah.

%set could simply be done without $&keeplexicalbinding, since settor functions are already not "lexically-aware". Having a general %set function rather than per-variable settors would allow:

1. Explicit interaction with the environment, enabling more flexibility around how variables are exported
2. A solution to the foo, set-foo, set-set-foo, set-set-set-foo problem, which enables (I think?) read-only variables, as well as better behavior in tracing variables in something like esdebug