local-expand

[gelisam]

We can imagine two versions of local-expand.

local-expand-syntax executes at phase 1, takes a Syntax object and expands it at phase 0, producing a Syntax object. Syntax objects can be deconstructed with open-syntax, spliced into larger Syntax objects with close-syntax, and eventually returned by the macro.

local-expand-core executes at phase 1, takes a Syntax object and expands it at phase 0, producing an opaque value of type Core. Values of type Core can be deconstructed carefully, producing well-formed Core sub-terms, they can be embedded into larger Syntax objects, and eventually returned by the macro.

One disadvantage of local-expand-syntax is that once the Syntax it returns is returned by the macro, the expander needs to expand it again, even though it is known to already contain only core constructs. This can lead to quadratic performance in some cases, and for this reason, we have long wanted Klister to have local-expand-core.

[gelisam]

I think it might make sense to have a variant of local-expand-syntax which only expands a single user macro for one step. Let's call it local-expand-single-step. For example, if (list 1) expands to (:: 1 (list)) and then to (:: 1 (nil)), it would help with examples and debugging to get (list 1) rather than (:: 1 (nil)) or its core representation.

[gelisam]

local-expand-core seems related to #119.

It seems unwise to expose Klister's definition of Core, that seems like an implementation detail which is likely to change and Klister is specifically designed to help the programmer write programs which do not depend on implementation details such as the order in which the type-checker and the expander perform their steps. However, once programmers are able to define their own domain-specific MyCore language, then it starts to make a lot of sense to call local-expand-my-core on a Syntax object in order to obtain a MyCore value whose possible constructors are well-known to the programmer.

[gelisam]

Here is a concrete use case. I would like to write a #lang whose type-checker is implemented via an ordinary function which recurs on a Core-like value (let's call it MyCore), to which the surface syntax of the language desugars (via macros of course). This could be implemented by a macro which takes a Syntax as input, and begins by calling local-expand-single-step to apply the desugarings until the head is the name of one of the MyCore constructors. At that point, the function would behave as if it pattern-matched on that constructor. The immediate children would still be Syntax values, on which the function can recur.

[gelisam]

Actually, that sounds like a step towards #119? Instead of f Identifier, the input and output are still Syntax, but they are now expected to be specifically-shaped Syntax which are headed by constructor names from a specific datatype. Let's call this approach "Syntax-headed regional macros", and let's call #119 "datatype-headed regional macros".

[gelisam]

Let's think about the implementation of local-expand-syntax. The expander for user-defined macros looks up the definition, evaluates the pure function applied to the call site's Syntax, and starts to execute the resulting Macro action. If that execution completes, a new task is forked for expanding the resulting Syntax; local-expand-syntax wants to return the resulting Syntax instead. If the execution gets stuck, a new task is forked for continuing the execution, and that task similarly forks-but-we-would-not on the resulting Syntax.

Given this, I can think of three approaches.

1. Duplicate the logic for expanding user-defined macros, so that the local-expand-syntax copy returns the resulting Syntax instead of forking a task.
2. Add a parameter (or equivalently, a defunctionalized continuation) specifying what to do with the resulting Syntax.
3. Change the logic to always return the Syntax, so now it's expanding regular user-defined macros which needs special treatment. For them, we could lean on our task system by expressing the continuation via a separate task which is blocked until that Syntax is known. That task would actually contain most of the expandOneForm logic: it would look at the head, handle all the primitive macros, add #% wrappers to literals, and it is only for user-defined macros that it would create a mortisse, fork the user-defined-macro expansion task to fill it, and fork itself to wait on the result.

I like (3), even though it is the most complicated and it creates extra temporary mortisses. I like the fact that it separates user-defined macros from primitive macros, and that it does so by looking at whether the head is the name of a primitive macro or the name of a user-defined macro. It looks like the Syntax-headed regional macro approach I just described, and thus it seems plausible that it could be generalized to regions other than Core, or to datatype-headed regional macros!