Scopes: Algorithm for computing original stack frames

[hbenl]

Preview

This is an initial draft of the algorithm for computing original stack frames (implemented here).
This PR makes use of GetOriginalPosition from #195.

TODO:

• handle multiple generated sources and sourcemaps
• handle outlined functions
• handle frames with generated locations that have no mapping

[szuend]

Thanks for writing up the algorithm! I had a cursory initial look.

IMHO ComputeOriginalFrames does potentially too much: It'll also have to handle inlining/outlining of frames. Together with building the scope chain for every frame such an abstract operation does a lot.

Rather then having one catch all abstract operation I suggest we spec out some basic building blocks. Concretely I suggest the following 3 "entry point operations":

• FindOriginalFunctionName(position: GeneratedPosition): string|null

  This is already somewhat in the current PR, albeit spread out. I'd spec it as:

  1. Find inner-most generated range for position (already there).
  2. Take the definition scope of the inner-most generated range
  3. Walk it outwards until we find a scope with isStackFrame = true.

• SymbolizeStackTrace(rawFrames: Array<{url: URL, position: GeneratedPosition}>): Array<{name?: string, position: OriginalPosition}>

  Doesn't handle scopes at all, purely translates a stack trace from generated source to original source. Note that rawFrames can contain frames across different bundles so we need some "implementation-defined" operation to get at different "Decoded source map" via a raw frames URL.

  This operation expands inlined frames, collapses outlined frames and utilizes FindOriginalFunctionName and GetOriginalPosition to translate frames.

• BuildScopeChain(position: GeneratedPosition): OriginalScopeWithValues[]

  Where OrigianlScopeWithValues is a record with:

  • The OriginalScope
  • A map of original variable name to the concrete JS value

  This operation does part of whats in ComputeOriginalFrame of the PR: Find the inner-most range and get it's definition scope. Walk that scope outwards and also find the corresponding generated range for each scope in the chain.

  I'd suggest though that we don't spec out how the evaluation works: Both Firefox and Chrome have the ability to evaluate expressions in the lexcial environment of a paused position but there might be restrictions. As such, I think it would ok to require a host-defined operation EvaluateInRange(range: GeneratedRange, expression: string): JSValue.

  We can leave it up to debuggers how to find the right debugger scope to evaluate expression in. We'll already spec how generated ranges relate to concrete ECMAScript grammar productions, so implementations should be able to then find the right debugger scope from that.

[hbenl]

Thanks for the feedback!

• FindOriginalFunctionName isn't suitable for inlining: in that case we start with 1 frame with a generatedPosition in the inlined function and need to generate 2 frames, so we need 2 function names. FindOriginalFunctionName(generatedPosition) will return the name of the inlined function but where do we get the name of the function that it was inlined into? The solution is to start with an OriginalPosition, which will be GetOriginalPosition(generatedPosition) for getting the inlined function name and GeneratedRange.callSite for getting the name of the function it was inlined into.
• BuildScopeChain also needs an originalPosition to handle inlining correctly (also set to either GetOriginalPosition(generatedPosition) or GeneratedRange.callSite for the 2 original frames) but it also still needs the generatedPosition for resolving subrange bindings.
• SymbolizeStackTrace would then also need to include OriginalPositions in its output so that those can be fed into BuildScopeChain. And it needs to specify the corresponding generated frame.
• I described how the evaluation works mostly because I want to keep the requirements that we place on debuggers as simple as possible. EvaluateWithBindings is simpler than EvaluateInRange IMHO, it's already available in Chrome and Firefox also has this capability, it just needs to be exposed in its debugger protocol. AFAIK neither browser can currently evaluate expressions in a parent of the current scope. I'd also prefer this part of the algorithm to be part of the spec rather than implementation-defined behavior.

[hbenl]

Note that rawFrames can contain frames across different bundles

Good point, my implementation and this PR currently work with only one generated source.

[szuend]

• Good point that FindOriginalFunctionName can return more than one name. Note though that it's kind of the special case of my SymbolizeStackTrace with only a single frame.
• Also good point that BuildScopeChain needs an original position. Alternatively we could pass in an "inlined frame index". The number of inlined frames is basically the length of the result of SymbolizeStackTrace with a single generated position.
• FWIW, Chrome doesn't have the possibility to pass custom bindings via protocol and this would also be non-trivial to implement. On the other hand, I have concrete plans to add a scopeIndex to the Debugger.evaluateOnCallFrame command, so expressions can be evaluated in an arbitrary scope of a call frame. That is why I'd prefer to keep this implementation-defined. Different engines handle "debug evaluate" or "evaluate in a lexical environment" differently and offer different capabilities, so I don't want to prescribe requirements that no or only half the implementation end up following.

[hbenl]

Chrome doesn't have the possibility to pass custom bindings via protocol

I think this can be emulated using Runtime.callFunctionOn.

[hbenl]

I have concrete plans to add a scopeIndex to the Debugger.evaluateOnCallFrame command, so expressions can be evaluated in an arbitrary scope of a call frame.

Nice!

I don't want to prescribe requirements that no or only half the implementation end up following

OK, we could use "evaluate in this scope" in the algorithm and then append a description how "evaluate in this scope" can be emulated using "evaluate with bindings".

[szuend]

Chrome doesn't have the possibility to pass custom bindings via protocol

I think this can be emulated using Runtime.callFunctionOn.

I'm not sure how'd you create the bindings object in the first place. You'd still need to read the values for generated variables off the stack or out of closure contexts. Debugger.evaluateOnCallFrame has this built-in by basically building with scopes for each scope that know how to read from the stack or closure contexts.

At least from a V8 and Chrome DevTools perspective it would be much easier to just extend evaluateOnCallFrame to allow expression evaluation in an arbitrary scope returned by the Debugger.paused event.

[hbenl]

I have added the suggested SymbolizeStackTrace and BuildScopeChain functions and changed ComputeOriginalFrames to use those.
Note that I have kept the Debugger Frame Record and Debugger Scope Record types (including the scope bindings) since they are used to return the original frames including their scopes but the algorithm now only uses the start/end positions from the generated Debugger Scope Records (to find the index of the Debugger Scope Record corresponding to a Generated Range Record) but not the bindings.