Inconsistent behavior of REPL

[p51lee]

Version

v18.11.0

Platform

Darwin MacBook-Pro.local 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:19:52 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T6000 arm64

Subsystem

No response

What steps will reproduce the bug?

There is an inconsistent behavior of nodejs REPL executing try { for ( await of [ ] ); } catch { } with respect to just executing test.js file.

// test.js
try {
  for ( await of [ ] );
} catch { }

$ node test.js
// no error

$ node
Welcome to Node.js v18.11.0.
Type ".help" for more information.
> try{ for ( await of [ ] ); } catch { }
Uncaught SyntaxError:
try{ for ( await of [ ] ); } catch { }
                      ^

Unexpected token ']'

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior?

No response

What do you see instead?

Running try{ for ( await of [ ] ); } catch { } with nodejs REPL throws SyntaxError but executing the test.js file using nodejs terminates without error.

Additional information

No response

[pecampelo]

Hey, I just read your issue, and before anyone else that is much more able than me goes ahead and has a look at you with it, I just want to clear my mind on a doubt I had, it could even be the case it solves it:

From my understanding and experience, for loops are usually synchronous and would not support awaiting of a Promise inside its iterator instantiation, only inside the loop itself. The only known case I am aware of that supports await with for loops is `for await. So, I am guessing you are purposefully attempting to make it throw a SyntaxError.

Assuming that's the case, first guess would be the use of both reserved words together, await and of. The word of is used most often with iterators...so...that might break compilation in REPL since it is in the same line (?) Have you tried pressing Enter to separate lines in the REPL?

That is a non-standard occurrence that I have been through some times in Chromium's console, as well.

[aduh95]

@pecampelo what this code is trying to do is to use await as an identifier (you can do var await = 'some value' (or simply await = 'some value' in non-strict mode) as long as you're outside an async function), so it has nothing to do with asynchronous flow in this case, it's just the name of the identifier (for(await of [1,2,3]) { console.log(await) } prints out 1, then 2, then 3).

FWIW none of the parsers in https://astexplorer.net/ are able to parse it correctly, and in the browsers world only Safari dev-tools seems to be able to parse it correctly (I've also tried with Firefox and Chromium, but it fails there).

Using --no-experimental-repl-await CLI flag solves the issue:

$ node
Welcome to Node.js v18.11.0.
Type ".help" for more information.
> for(await of [1,2,3])  { console.log(await) }
Uncaught SyntaxError: 
for(await of [1,2,3])  { console.log(await) }
                    ^

Unexpected token ')'
$ node --no-experimental-repl-await
Welcome to Node.js v18.11.0.
Type ".help" for more information.
> for(await of [1,2,3])  { console.log(await) }
1
2
3
undefined

/cc @guybedford

[pecampelo]

Interesting. I would refrain from using reserved words but I get your point, that was a very deep explanation on what the compiler can or cannot parse. Thanks for the elucidation

[ErickWendel]

Should we close this issue as it's not a problem?

[aduh95]

I mean, it is a bug, it might not be a problem for you but it is for the OP, I don't think we should discard it. That being said, the issue is on acorn (which we use to parse the input on the REPL when using --experimental-repl-await, enabled by default), there's not much that can be done in this repo. Do we know if it has been reported to them?

[cola119]

for(await of [1,2,3]) { console.log(await) } is valid syntax in terms of CommonJS but is not valid in terms of ES modules. I'm not sure if the REPL is running on ES modules mode, but if so, this is not a bug?

$ node --input-type=commonjs -e "for(await of [1,2,3]) { console.log(await) }"
1
2
3

$ node --input-type=module -e "for(await of [1,2,3]) { console.log(await) }"
SyntaxError: Unexpected token ')'

Currently, when a given code contains an await, the REPL transforms it to (async () => { <original code> })() to emulate top-level await. for(await of [1,2,3]) { console.log(await) } is a valid syntax in terms of CommonJS but the transformed code (async () => { for(await of [1,2,3]) { console.log(await) } })() is no longer valid, which causes this inconsistent behavior, so this is not a bug from acorn side.

[aduh95]

for(await of [1,2,3]) { console.log(await) } is valid syntax in terms of CommonJS but is not valid in terms of ES modules. I'm not sure if the REPL is running on ES modules mode, but if so, this is not a bug?

CJS vs ESM is [a bit of a red herring here]¹, what you show with your example is the difference between strict mode vs loose mode. It's defined in the ECMAScript spec (13.1.1) that await cannot be used as an identifier in strict mode. ESM contains "module code", which as defined in 11.2.2 is always strict mode. I'm not sure if REPL would qualify as "global code" or "eval code" (and it certainly is not "module code", otherwise we wouldn't need hacks to make top-level await work), but for sure it's not strict by default .

(Worth noting that JS engines are not always spec compliant btw, e.g. V8 sometimes lets you use await as an identifier in strict mode).

this is not a bug from acorn side.

Alright, I'd argue there's still a bug on acorn side as they fail to parse for(await of [1,2,3]); as valid JS. Granted it's not related to the error we see here.

Interestingly, our fallback is to remove the awaits to see if the code is runnable, maybe that's a mistake and we should try to run the code as is instead:

node/lib/repl.js

Lines 456 to 473 in a01dbf1

	// Remove all "await"s and attempt running the script
	// in order to detect if error is truly non recoverable
	const fallbackCode = RegExpPrototypeSymbolReplace(/\bawait\b/g, code, '');
	try {
	vm.createScript(fallbackCode, {
	filename: file,
	displayErrors: true,
	importModuleDynamically: (specifier, _, importAssertions) => {
	return asyncESM.esmLoader.import(specifier, parentURL,
	importAssertions);
	}
	});
	} catch (fallbackError) {
	if (isRecoverableError(fallbackError, fallbackCode)) {
	recoverableError = true;
	err = new Recoverable(e);
	}
	}

Footnotes

1. But also maybe not, the spec does add more strictness with regard to the use of await and yield when the parse goal is Module instead of Script. It's unclear to me if this fails under one of the defined exceptions. ↩

[BridgeAR]

This would be fixed when #8309 is implemented. The repl mode provided also makes sure proper top level await is available.