go/types: panic in allowVersion if token.Pos is out of bounds

[adonovan]

The type checker performs various checks on the Go language level applicable to certain constructs, such as whether underscores are permitted in integer literals, which requires go1.13+. The logic to extract the file version is fragile, and panics if the position information is out of bounds, as is common when type checking synthesized or modified code.

Here's a reproducible test case:

https://go.dev/play/p/1GF5wfNIBYw

	// Parse the file.
	fset := token.NewFileSet()
	f, _ := parser.ParseFile(fset, "a.go", "package p; const k = 123", 0)
	format.Node(os.Stderr, fset, f)

	// Set an invalid Pos on the BasicLit.
	ast.Inspect(f, func(n ast.Node) bool {
		if lit, ok := n.(*ast.BasicLit); ok {
			lit.ValuePos = 99999
		}
		return true
	})

	// Type check
	pkg := types.NewPackage("p", "p")
	check := types.NewChecker(&types.Config{}, fset, pkg, nil)
	if err := check.Files([]*ast.File{f}); err != nil { // panic: file not found for pos = 99999
		log.Fatal(err)
	}

The type checker should not panic due to missing or inaccurate position information. (Ideally it wouldn't use such information for type checking judgements, only for symbol positions and error messages, but that ship has sailed.)

A more robust solution would be to propagate the file version information top-down when traversing over the syntax trees, instead of trying to figure out which file a given node belongs to based on its Pos.

This is the root cause of the gopls crashes #69338, and I have encountered it in other refactoring work.

[gabyhelp]

Related Issues and Documentation

• x/tools/gopls: investigate if 'fixed' syntax needs more accurate positions #64335
• x/tools/gopls: "file not found for pos = %d (%s)" crash in go/types #69338
• go/types2, types2: version check (Checker.allowVersion) ignores module version if position information is missing #66274 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[gopherbot]

Change https://go.dev/cl/613735 mentions this issue: go/types: compute effective Go version independent of token.Pos