Timeout behaves differently between scanner and batch scanner

[keith-turner]

Describe the bug

Made the following change to TimeoutIT to copy the existing batch scanner timeout test and create a similar version for the scanner. The new scanner version is not timing out.

diff --git a/test/src/main/java/org/apache/accumulo/test/functional/TimeoutIT.java b/test/src/main/java/org/apache/accumulo/test/functional/TimeoutIT.java
index fdbfcd820d..09649d0434 100644
--- a/test/src/main/java/org/apache/accumulo/test/functional/TimeoutIT.java
+++ b/test/src/main/java/org/apache/accumulo/test/functional/TimeoutIT.java
@@ -32,6 +32,7 @@ import org.apache.accumulo.core.client.BatchWriter;
 import org.apache.accumulo.core.client.BatchWriterConfig;
 import org.apache.accumulo.core.client.IteratorSetting;
 import org.apache.accumulo.core.client.MutationsRejectedException;
+import org.apache.accumulo.core.client.Scanner;
 import org.apache.accumulo.core.client.TimedOutException;
 import org.apache.accumulo.core.data.Mutation;
 import org.apache.accumulo.core.data.Range;
@@ -48,9 +49,10 @@ public class TimeoutIT extends AccumuloClusterHarness {
   @Test
   public void run() throws Exception {
     try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
-      String[] tableNames = getUniqueNames(2);
+      String[] tableNames = getUniqueNames(3);
       testBatchWriterTimeout(client, tableNames[0]);
       testBatchScannerTimeout(client, tableNames[1]);
+      testScannerTimeout(client, tableNames[2]);
     }
   }
 
@@ -104,4 +106,31 @@ public class TimeoutIT extends AccumuloClusterHarness {
     }
   }
 
+  public void testScannerTimeout(AccumuloClient client, String tableName) throws Exception {
+    client.tableOperations().create(tableName);
+
+    try (BatchWriter bw = client.createBatchWriter(tableName)) {
+      Mutation m = new Mutation("r1");
+      m.put("cf1", "cq1", "v1");
+      m.put("cf1", "cq2", "v2");
+      m.put("cf1", "cq3", "v3");
+      m.put("cf1", "cq4", "v4");
+      bw.addMutation(m);
+    }
+
+    try (Scanner scanner = client.createScanner(tableName)) {
+      scanner.setRange(new Range());
+
+      // should not timeout
+      scanner.forEach((k, v) -> {});
+
+      scanner.setTimeout(5, TimeUnit.SECONDS);
+      IteratorSetting iterSetting = new IteratorSetting(100, SlowIterator.class);
+      iterSetting.addOption("sleepTime", 6000 + "");
+      scanner.addScanIterator(iterSetting);
+
+      assertThrows(TimedOutException.class, () -> scanner.iterator().next(),
+          "scanner did not time out");
+    }
+  }
 }

Expected behavior

The javadoc for the setTimeout method states : This setting determines how long a scanner will automatically retry when a failure occurs. By default, a scanner will retry forever. In the existing batch scanner test there is no error (only a long running scan that never returns data), but it still times out. Need to determine if the scanner should match the batch scanner behavior or the javadoc. The scanner impl does have timer internally, however its continually reset and never causes a timeout for the case of a long running scan not returning data. Not sure if the timer works for the error case or not.