TestBPReorderingMergePolicy fails CheckIndex.testHnswGraph

[msokolov]

Description

TestBPReorderingMergePolicy > testReorderOnAddIndexes FAILED
    org.apache.lucene.index.CheckIndex$CheckIndexException: Field "vector" has repeated neighbors of node 6978 with value 7004
        at __randomizedtesting.SeedInfo.seed([B1FA21A9AAC314F2:A0D1EE47A3A0C942]:0)
        at app/[email protected]/org.apache.lucene.index.CheckIndex.testHnswGraph(CheckIndex.java:2931)
        at app/[email protected]/org.apache.lucene.index.CheckIndex.testHnswGraphs(CheckIndex.java:2806)
        at app/[email protected]/org.apache.lucene.index.CheckIndex.testSegment(CheckIndex.java:1125)
        at app/[email protected]/org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:822)
        at app/[email protected]/org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:592)
        at app/[email protected]/org.apache.lucene.tests.util.TestUtil.checkIndex(TestUtil.java:338)
        at app/[email protected]/org.apache.lucene.tests.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:921)
        at app/[email protected]/org.apache.lucene.util.IOUtils.close(IOUtils.java:85)
        at app/[email protected]/org.apache.lucene.util.IOUtils.close(IOUtils.java:72)
        at app//org.apache.lucene.misc.index.TestBPReorderingMergePolicy.testReorderOnAddIndexes(TestBPReorderingMergePolicy.java:241)

B1FA21A9AAC314F2

Gradle command to reproduce

./gradlew :lucene:misc:test --tests "org.apache.lucene.misc.index.TestBPReorderingMergePolicy" -Dtests.seed=B1FA21A9AAC314F2

[msokolov]

Disabling HnswGraphBuilder.connectComponents makes the test case pass. I suspect that connectComponents is not careful about adding duplicates when making its new links bidirectional? It's benign, but the new CheckIndex check doesn't like it. We should probably (1) make CheckIndex more lenient, (2) fix connectComponents() so it doesn't create duplicates, and (3) re-enable the check, with an index version gate (since there may be indexes in the wild with duplicates now)

  void finish() throws IOException {
    // System.out.println("finish " + frozen);
    // connectComponents();
    frozen = true;
  }