Previewing input

[randomeizer]

Hi there,

I've been using the Parsing library in a couple of projects now, and the need to check on parsed content has come up a couple of times. I've pitched Not and Peek elsewhere related to this.

One concrete example from the project is parsing 'identifiers' (eg, function/variable/names).

struct Identifier {
    let value: Substring

    init(_ value: Substring) {
        self.value = value
    }
}

Basically the rule is that the first character may be a letter or an underscore, and subsequent characters may be letters, underscores or digits. So, my first attempt was this:

let identifier = Parse {
    Prefix(1) { $0.isNumber || $0 == "_" }
    Prefix { $0.isNumber || $0 == "_" || $0.isLetter }
}
.map { Identifier($0.append(contentsOf: $1)) }

It works, but feels a bit clunky having to append the two chunks of Substring to achieve it. So, with Not and Peek added, it is currently this:

let identifier = Parse(Identifier.init) {
    Skip {
        Peek { Prefix(1) { $0.isNumber || $0 == "_" } }
    }
    Prefix { $0.isNumber || $0 == "_" || $0.isLetter }
}

Which is better, but requiring a Skip feels somewhat redundant/noisy. While it makes sense for Peek to return the value, it would be nice if there was another operator available that would confirm without returning the duplicated character(s).

nom has a verify function which might be handy here. An example implementation could be:

let identifier = Verify(Identifier.init) {
    Prefix { $0.isNumber || $0 == "_" || $0.isLetter }.map(String.init)
} matches: {
   $0.first.map { c in c == "_" || c.isLetter } ?? false
}

Alternately, a variation on Peek (lets call it Require for now) which always returns Void if the Upstream Parser is successful would allow us to omit the Skip:

let identifier = Parse(Identifier.init) {
    Require { Prefix(1) { $0.isNumber || $0 == "_" } }
    Prefix { $0.isNumber || $0 == "_" || $0.isLetter }
}

There isn't really an analogue for this operator in nom, and I'm not sure if Require is the right name. Other possibilities might be Match, Has, Check, or Expect.

Essentially, it's accounting for the scenario where you want to check the beginning of your input, but include that as part of a longer section of input to pass on to the next step. Verify lets you do that (and more), but is not optimised for that case. Prefix lets you do it, but you have to Skip sometimes, depending on how your upstream Parser works.

Thoughts on the concept in general? I think Verify is probably worth adding regardless, for doing more complicated post-parse checking. But I think it's kind of overkill for this purpose, and leads to some additional hoop-jumping in this context.

[mbrandonw]

nom has a verify function which might be handy here. An example implementation could be:

let identifier = Verify(Identifier.init) {
    Prefix { $0.isNumber || $0 == "_" || $0.isLetter }.map(String.init)
} matches: {
   $0.first.map { c in c == "_" || c.isLetter } ?? false
}

I believe this is equivalent to our filter method if that helps at all:

let identifier = Parse(Identifier.init) {
  Prefix { $0.isNumber || $0 == "_" || $0.isLetter }.map(String.init)
}
.filter {
  $0.first.map { c in c == "_" || c.isLetter } ?? false
}

[randomeizer]

Hmm, interesting. I keep forgetting there are method modifiers on Parsers. It would definitely be useful for more complex cases.

What are your thoughts on the value of something like Require vs Skip(Prefix(...))? Personally, I'm more likely to skip the result than use it, but I may be the exception.

[randomeizer]

I've pushed a PR with an implementation of Expect, which returns Void if the upstream parser succeeds, nil if not, and neither consumes the input.

Thoughts?