How to collect N count the right way?

[ytyubox]

Hi, Thanks for this great tool, I really enjoy developing with it.

I found a confusing case in PrefixTests

swift-parsing/Tests/ParsingTests/PrefixTests.swift

Lines 11 to 15 in 4d80541

	func testPrefixUnder() {
	var input = "42 Hi!"[...]
	XCTAssertThrowsError(try Prefix(8).parse(&input))
	XCTAssertEqual("", input)
	}

When the parse fails with an error, the input is gone. Would you happen to know if this is expected?

Unlike BoolTests, the input remains the same as the parse failed.

swift-parsing/Tests/ParsingTests/BoolTests.swift

Lines 17 to 31 in 4d80541

	func testParseFailure() {
	var input = "Hello, world!"[...].utf8
	XCTAssertThrowsError(try Bool.parser().parse(&input)) { error in
	XCTAssertEqual(
	"""
	error: unexpected input
	--> input:1:1
	1 | Hello, world!
	| ^ expected "true" or "false"
	""",
	"\(error)"
	)
	}
	XCTAssertEqual("Hello, world!", Substring(input))
	}

The context I expect is to get Int first then collect the next N String. I think I can use Prefix(Int) as follow, but maybe I approach the wrong way. 😞
The test case of 4d80541

func test() throws {
  var input = "3xyz"[...].utf8
  let item = Parse {
    Int.parser()
      .flatMap { count in
        Prefix(count)
      }
  }
  let i = try item.parse(&input)

  XCTAssertEqual(Substring(i), "xyz")
  XCTAssertEqual(Substring(input), "")
}

func test_failure() {
  var input = "3xy"[...].utf8
  let item = Parse {
    Int.parser()
      .flatMap { count in
        Prefix(count)
      }
  }

  XCTAssertThrowsError(try item.parse(&input))
  XCTAssertEqual("", Substring(input)) // 🤯 I was expecting the input remains untouched.
}

Maybe collecting N count have a better approach than using Prefix? 🤔

[ytyubox]

The patch in my head for FlatMap and Prefix

Click to expand

diff --git a/Sources/Parsing/ParserPrinters/Prefix.swift b/Sources/Parsing/ParserPrinters/Prefix.swift
index ebc24ca2b..65a378c0d 100644
--- a/Sources/Parsing/ParserPrinters/Prefix.swift
+++ b/Sources/Parsing/ParserPrinters/Prefix.swift
@@ -101,18 +101,21 @@ public struct Prefix<Input: Collection>: Parser where Input.SubSequence == Input
   @inlinable
   @inline(__always)
   public func parse(_ input: inout Input) throws -> Input {
+    let copied = input
     var prefix = self.maximum.map(input.prefix) ?? input
     prefix = self.predicate.map { prefix.prefix(while: $0) } ?? prefix
     let count = prefix.count
     input.removeFirst(count)
     guard count >= self.minimum else {
+      let errorInput = input
+      input = copied
       let atLeast = self.minimum - count
       throw ParsingError.expectedInput(
         """
         \(self.minimum - count) \(count == 0 ? "" : "more ")element\(atLeast == 1 ? "" : "s")\
         \(self.predicate == nil ? "" : " satisfying predicate")
         """,
-        at: input
+        at: errorInput
       )
     }
     return prefix
diff --git a/Sources/Parsing/Parsers/FlatMap.swift b/Sources/Parsing/Parsers/FlatMap.swift
index b41aacd0a..1abc807b1 100644
--- a/Sources/Parsing/Parsers/FlatMap.swift
+++ b/Sources/Parsing/Parsers/FlatMap.swift
@@ -45,11 +45,13 @@ extension Parsers {
       do {
         return try self.transform(self.upstream.parse(&input)).parse(&input)
       } catch let ParsingError.failed(reason, context) {
+        let errorInput = input
+        input = original
         throw ParsingError.failed(
           reason,
           .init(
             originalInput: original,
-            remainingInput: input,
+            remainingInput: errorInput,
             debugDescription: context.debugDescription,
             underlyingError: ParsingError.failed(reason, context)
           )
diff --git a/Tests/ParsingTests/ConditionalTests.swift b/Tests/ParsingTests/ConditionalTests.swift
index b655c9856..72985b98e 100644
--- a/Tests/ParsingTests/ConditionalTests.swift
+++ b/Tests/ParsingTests/ConditionalTests.swift
@@ -30,7 +30,7 @@ final class ConditionalTests: XCTestCase {
         "\(error)"
       )
     }
-    XCTAssertEqual(" Hello, world!", input)
+    XCTAssertEqual("43 Hello, world!", input)
   }
 }
 
diff --git a/Tests/ParsingTests/FlatMapTests.swift b/Tests/ParsingTests/FlatMapTests.swift
index c7911d7b7..c42a4ada6 100644
--- a/Tests/ParsingTests/FlatMapTests.swift
+++ b/Tests/ParsingTests/FlatMapTests.swift
@@ -43,6 +43,6 @@ final class FlatMapTests: XCTestCase {
         "\(error)"
       )
     }
-    XCTAssertEqual("llo, world!", Substring(input))
+    XCTAssertEqual("Hello, world!", Substring(input))
   }
 }
diff --git a/Tests/ParsingTests/PrefixTests.swift b/Tests/ParsingTests/PrefixTests.swift
index 70fbace0a..9279be539 100644
--- a/Tests/ParsingTests/PrefixTests.swift
+++ b/Tests/ParsingTests/PrefixTests.swift
@@ -27,7 +27,7 @@ final class PrefixTests: XCTestCase {
         "\(error)"
       )
     }
-    XCTAssertEqual("", input)
+    XCTAssertEqual("42 Hi!", input)
   }
 
   func testPrefixWhile() {
@@ -61,7 +61,7 @@ final class PrefixTests: XCTestCase {
         "\(error)"
       )
     }
-    XCTAssertEqual("", input)
+    XCTAssertEqual("42 Hello, world!", input)
   }
 
   func testPrefixRangeFromWhileSuccess() {

[tgrapperon]

Hey @ytyubox,

When the parse fails with an error, the input is gone. Would you happen to know if this is expected?

Yes, this is expected. For performance reasons, parsers are not backtracking by default (that is restoring the input if they fail to consume completely the input). You can find more information about this behavior in the documentation.

Only a handful of them are effectively backtracking, among them the Parse parser. So in your case, you should wrap your Prefix parser into a Parse parser to behave as you'd expect. The Bool parser is coincidently not consuming the input when it fails, but this is an implementation detail.

Correction: The Parse parser isn't backtracking either. You should consider the input you pass to the initial parser as a throwaway value that can be partially consumed whatever the parsing result is. As an alternative, you can wrap the parser in a OneOf that backtracks, or use a .parse(…) method that doesn't use an inout argument.

[ytyubox]

Hi, @tgrapperon, Thank you! I found a relative thread about it, #232.
Interestingly, I require a way not to consume the input for buffering the input.
This performance feature is excellent, and I want to keep it. So I can come up with a decorator.

extension Parser { // Or a result builder, maybe.
  func alwaysRevertOnCatch() -> AlwaysRevertParser<Self> {
    .init(parser: self)
  }
}

struct AlwaysRevertOnCatchParser<P: Parser>: Parser {
  let parser: P
  typealias Input = P.Input
  typealias Output = P.Output

  func parse(_ input: inout P.Input) throws -> P.Output {
    let originInput = input
    do {
      return try parser.parse(&input)
    } catch {
      input = originInput
      throw error
    }
  }
}

It can be an extension of my project, but I would like to keep studying more about backtracking and other performance topics.

Thanks a lot!

[tgrapperon]

Great! I'm wondering if this wouldn't be equivalent to OneOf { someParser }, as this one backtracks.
IRRC, but not useful in your case, Not is also notoriously backtracking.

[tgrapperon]

I'm wondering if this wouldn't be equivalent to OneOf { someParser }

It isn't! OneOf is not backtracking when only one parser is specified, as it builds as the parser itself. Maybe the behavior should be homogenized, or the documentation of OneOf amended.

[mbrandonw]

I just opened this PR #287 that will hopefully make this stuff a lot clearer. What do y'all think?