Difficulty with packet descriptor

[JetForMe]

Hi. I'm trying to parse packets that can either be one of four 1-byte packets, or a variable-length packet. That is, a byte at the start of a packet of 0x06 (ACK), 0x15 (NAK), 0x16 (SYN), or 0x18 (CAN) constitutes a complete message. A starting byte of 0x01 (SOF) means at least four more bytes are coming.

When I send a request, I get back an ACK byte, followed by a SOF frame. If my request was bad, I just get back a NAK byte.

The full set of bytes I get from the other end looks like this:

 0: 06 01 10 01   15 5a 2d 57   61 76 65 20   34 2e 35 34
10: 00 01 93

What that dump shows is the ACK, followed by a SOF frame of 18 bytes: SOF (0x01), length (0x10), type (0x01), some string data, 0x00, 0x01, and a checksum of 0x5D. Because I never ACK the response, it's repeated, so the 0x93 above is followed by 0x01, 0x10, 0x01, 0x15.

My packet descriptor evaluator looks like this. Unfortunately, it never makes it to the the checksum validation. Instead, it seems to decide the length is 0x93, which suggests to me the packet parser is calling my function after skipping some bytes. I would have thought it would keep sending me the same buffer, with new bytes appended to the end, so I could keep reevaluating them until I got a good packet.

let pd = ORSSerialPacketDescriptor(maximumPacketLength: 256, userInfo: nil)
{ (inData) -> Bool in
    
    //  Is there any data?
    
    guard
        let data = inData,
        data.count > 0
    else
    {
        return false
    }
    
    //  Is it one of the single-byte frames?
    
    let singles: [FrameStart] = [.ack, .nak, .syn, .can]
    if data.count == 1,
        let start = FrameStart(rawValue: data[0]),
        singles.contains(start)
    {
        return true
    }
    
    //  It’s not, see if it’s a SOF frame…
    
    if data.count < 5 { return false }                              //  These frames are at least 5 bytes
    if data[0] != FrameStart.sof.rawValue { return false }          //  Not a SOF frame
    if data[2] != 0x00 && data[2] != 0x01 { return false }          //  Not a request or response
    let length = data[1]
    if data.count != length + 2                                     //  Wrong length
    {
        return false
    }
    
    //  Validate the checksum…
    
    let checksum = data[1 ..< data.count - 1].reduce(0xff, ^)
    if checksum != data.last! { return false }
    
    return false
}

[StateMachineJunkie]

Did you ever figure this out? I think I'm seeing a related problem. I too expect the response evaluator to return with a progressively growing buffer, which is collecting data as it is coming in from the serial port. Instead, with each call of the evaluator, I'm getting only partial data. My serialPort(_:didReceive) method actually show a complete response in the buffer. That response looks like this: Received "2.1.0.2
" (32 bytes)

However, my response evaluator is invoked multiple times with the following byte counts (associated with each Data value passed in):
Data = 1 bytes

Data = 1 bytes
Data = 2 bytes

Data = 1 bytes
Data = 2 bytes
Data = 3 bytes

Data = 1 bytes
Data = 2 bytes
Data = 3 bytes
Data = 4 bytes

Data = 1 bytes
Data = 2 bytes
Data = 3 bytes
Data = 4 bytes
Data = 5 bytes

Data = 1 bytes
Data = 2 bytes
Data = 3 bytes
Data = 4 bytes
Data = 5 bytes
Data = 6 bytes

Data = 1 bytes
Data = 2 bytes
Data = 3 bytes
Data = 4 bytes
Data = 5 bytes
Data = 6 bytes
Data = 7 bytes

Data = 1 bytes
Data = 2 bytes
Data = 3 bytes
Data = 4 bytes
Data = 5 bytes
Data = 6 bytes
Data = 7 bytes
Data = 8 bytes

Data = 1 bytes
Data = 2 bytes
Data = 3 bytes
Data = 4 bytes
Data = 5 bytes
Data = 6 bytes
Data = 7 bytes
Data = 8 bytes
Data = 9 bytes

Data = 1 bytes
. . .
Data = 31 bytes

This continues all the way up to 31, one byte shy of the entire response length. You may also notice the progressing pattern in the byte counts. I did not notice this until just now. It is a definite clue to what is going on in the logic that invokes the response evaluator. It may be that I need to re-read the documentation but this is not the kind of behavior that is intuitive for an API like this.

[armadsen]

I should probably clarify the documentation around this. The behavior you're seeing is correct (minus not seeing all 32 bytes). The response evaluator is called repeatedly with progressively bigger chunks of the received data working backward. I agree this can be a bit confusing when you first encounter it, and I might design the API differently were I designing it today. But there is a good reason for it, as this approach is very robust in the face of occasional garbage data or incomplete packets, as well as fully supporting multiple (even overlapping) packet listeners being installed at the same time.

Anyway, it means that your response evaluator should check for both the beginning and end of the packet you're expecting (and/or the overall length, a matching checksum, etc., as is appropriate for your packet format). It should disqualify packets as efficiently as possible. And this API isn't necessarily a great choice if your packet format isn't very structured, ie. it's hard to write general purpose "is this a valid packet?" code.

That said, I'm concerned that you're only seeing 31/32 bytes. You should see all 32. @StateMachineJunkie feel free to email me privately if you want me to help you dig in a bit more.