workspace/tests returns unnecessarily disambiguated test items

[plemarquand]

Swift version

swift-driver version: 1.120.4 Apple Swift version 6.1 (swiftlang-6.1.0.108.3 clang-1700.0.11.1

Platform

macOS

Editor

Visual Studio Code

Description

When you have more than one XCTest test in an extension of an XCTestCase, the workspace/tests request will incorrectly return XCTests with the filename and line appended to the end of the ID for all tests in the extension. This only happens with background indexing enabled.

This disambiguation is really only supposed to be for swift-testing tests that share the same ID signature (i.e. @Test foo(_ x:Int) and @Test foo(_ x:String), which both have an id of foo(_:)).

Steps to Reproduce

Add the following test to WorkspaceTestDiscoveryTests.swift and run it:

func testXCTestExtensionWithMultipleMethodsAndBackgroundIndexing() async throws {
  let project = try await SwiftPMTestProject(
    files: [
      "Tests/MyLibraryTests/MyTests.swift": """
      import XCTest
      1️⃣final class MyTests: XCTestCase {}6️⃣

      extension MyTests {
        2️⃣func testOne() {}3️⃣
        4️⃣func testOneTwo() {}5️⃣
      }
      """,
    ],
    manifest: packageManifestWithTestTarget,
    enableBackgroundIndexing: true
  )

  let tests = try await project.testClient.send(WorkspaceTestsRequest())
  XCTAssertEqual(
    tests,
    [
      TestItem(
        id: "MyLibraryTests.MyTests",
        label: "MyTests",
        location: try project.location(from: "1️⃣", to: "6️⃣", in: "MyTests.swift"),
        children: [
          TestItem(
            id: "MyLibraryTests.MyTests/testOne()",
            label: "testOne()",
            location: try project.location(from: "2️⃣", to: "3️⃣", in: "MyTests.swift")
          ),
          TestItem(
            id: "MyLibraryTests.MyTests/testOneTwo()",
            label: "testOneTwo()",
            location: try project.location(from: "4️⃣", to: "5️⃣", in: "MyTests.swift")
          )
        ]
      )
    ]
  )
}

This produces a TestItem with two children that have IDs MyLibraryTests.MyTests/testOne():MyTests.swift:4:4 MyLibraryTests.MyTests/testOneTwo():MyTests.swift:5:4

Expected:
MyLibraryTests.MyTests/testOne() MyLibraryTests.MyTests/testOneTwo()

Logging

No response

[plemarquand]

I think this tracks back to an issue with the unit test SymbolOccurences returned from IndexStoreDB. In the test example above, three semanticTestSymbolOccurrences are returned from index?.unitTests().filter { return $0.canBeTestDefinition } ?? [].

Decisions about how to structure the hierarchy of these three results are made based off the relations of the symbols and the roles of these relations. Inspecting the result of semanticTestSymbolOccurrences shows that all the XCTests tests defined in an extension have a relation to the first test defined in the extension, and each relation has the role of childOf.

po semanticTestSymbolOccurrences.map({ "\($0.symbol.usr) -> \($0.relations.first?.symbol.usr ?? "nil")" })
▿ 3 elements
  - 0 : "c:@M@MyLibraryTests@objc(cs)MyTests -> nil"
  - 1 : "c:@CM@MyLibraryTests@objc(cs)MyTests(im)testOne -> s:e:c:@CM@MyLibraryTests@objc(cs)MyTests(im)testOne"
  - 2 : "c:@CM@MyLibraryTests@objc(cs)MyTests(im)testTwo -> s:e:c:@CM@MyLibraryTests@objc(cs)MyTests(im)testOne"

I'd expect this relation to point to c:@M@MyLibraryTests@objc(cs)MyTests and not s:e:c:@CM@MyLibraryTests@objc(cs)MyTests(im)testOne

This causes the test at index [2] to be parented to the test at index [1] instead of [0]. Ultimately when the tests have all been reparented we end up with:

po testsFromSemanticIndex.map({ "\($0.testItem.id) -> \($0.testItem.children.map(\.id))" })
▿ 3 elements
  - 0 : "MyTests -> []"
  - 1 : "MyTests -> ["MyTests/testOne()", "MyTests/testTwo()"]"
  - 2 : "MyTests -> ["MyTests/testOne()", "MyTests/testTwo()"]"

Instead of a single root element with testOne and testTwo as children.

[ahoppen]

Synced to Apple’s issue tracker as rdar://145759210