Proposal: Split operation and messageType into separate members of a message

[benfrancis]

This proposal builds on @hspaay's message envelope proposal in #34 (and #32), and @RobWin's suggestion for acknowledgements of certain messages (e.g. #29).

It is a further exploration of the idea of splitting operation and messageType into separate members of a message, where operation is an op name taken directly from the Thing Description specification and messageType is one of three basic types: request, response and notification.

I've kept response and notification separate as per @hspaay's original proposal because I do think the two are meaningfully different, though I still feel a bit uneasy about this because "notification" is not a clearly defined concept in existing WoT specifications. It also kind of mixes up the common request/response and publish/subscribe design patterns.

Although I have adopted the proposal of a consistent "name" member to contain the name of the affordance the message relates to, one way this proposal differs from @hspaay's proposal is that it keeps separate members for different data payloads depending on the type of operation, rather try to combine everything into input and output. This is intended to make the mapping of data schemas from a Thing Description more obvious and not have members with the same name but different meanings:

• Property-related messages have a "value" (conforming to the data schema of PropertyAffordances)
• Action related messages have an "input" and an "output" (conforming to the input and output data schemas in ActionAffordances) and a "status" (consistent with the HTTP Basic Profile)
• Event related messages have event "data" (conforming to the data member in EventAffordances)

Another way that this proposal differs is that there are never multiple responses to a request, only multiple notifications and a single response. This is achieved by property observations and event subscriptions having a request, response then notifications, and action invocations having a request, notifications then a single response.

Message Types

• request
• response
• notification

Operations

Operation	Lifecycle
readproperty	request, response
writeproperty	request, response
observeproperty	request, response, notification, notification...
unobserveproperty	request, response
readallproperties	request, response
writeallproperties	request, response
readmultipleproperties	request, response
writemultipleproperties	request, response
observeallproperties	request, response, notification, notification...
unobserveallproperties	request, response
invokeaction	request, notification, notification... response *
queryaction	request, response
queryallactions	request, response
subscribeevent	request, response, notification, notification...
unsubscribeevent	request, response
subscribeallevents	request, response, notification, notification...
unsubscribeallevents	request, response

* I'm a bit uneasy about the lifecycle of the invokeaction operation because we have kind of invented a combined invokeaction/observeaction operation, which doesn't exist in the Thing Description specification. There is currently no way to specify the data schema of an action status and no way to observe the status of a property in the the TD spec.

Examples

readproperty

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "readproperty",
  "name": "on",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Success Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "readproperty",
  "name": "on",
  "value": true,
  "timestamp": "2024-01-13T23:20:50.52Z",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Error Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "readproperty",
  "name": "on",
  "error": {
    "status": "404",
    "type": "https://w3c.github.io/web-thing-protocol/errors#not-found",
    "title": "Not Found"
    "detail": "No property found with the name 'on'"
  }
  "timestamp": "2024-01-13T23:20:50.52Z",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

writeproperty

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "adbefb2c-9c10-4a9e-b2d5-840a58ab667e",
  "messageType": "request",
  "operation": "writeproperty",
  "name": "on",
  "value": true,
  "correlationID": "b737e900-2b34-4315-bf9e-9eec1a0406c0"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "writeproperty",
  "name": "on",
  "value": true,
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Note: To address the issue raised in #38 about non-confirmable writes, a value could only be provided in a response if it is confirmed to have been written.

observeproperty

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "observeproperty",
  "name": "on",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "observeproperty",
  "name": "on",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Notification

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "notification",
  "operation": "observeproperty",
  "name": "on",
  "value": true,
  "timestamp": "2024-01-13T23:20:50.52Z",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

unobserveproperty

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "unobserveproperty",
  "name": "on",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "unobserveproperty",
  "name": "on",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

readallproperties

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "readallproperties",
  "name": "on",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "readpallproperties",
  "name": "on",
  "values": {
    "on": true,
    "level": 50
  },
  "timestamp": "2024-01-13T23:20:50.52Z",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

readmultipleproperties

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "readmultipleproperties",
  "names": ["on", "level"],
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "readmultipleproperties",
  "name": "on",
  "values": {
    "on": true,
    "level": 50
  },
  "timestamp": "2024-01-13T23:20:50.52Z",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

invokeaction

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "invokeaction",
  "name": "fade",
  "input": {
    "level": 100,
    "duration": 5
  }
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Notification

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "notification",
  "operation": "invokeaction",
  "name": "fade",
  "status": "running",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "response",
  "operation": "invokeaction",
  "name": "fade",
  "output": true,
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

queryaction

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "queryaction",
  "name": "fade",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "response",
  "operation": "queryaction",
  "name": "fade",
  "status": "completed",
  "output": true,
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

queryallactions

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "queryallactions",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "response",
  "operation": "queryallactions",
  "statuses": {
    "fade": {
      [
         {
          "status": "pending",
          "correlationID": "542e4567-e89b-12d3-a456-631968",
          "timeRequested": "2024-11-11T11:43:20.135Z"
        },
        {
          "status": "completed",
          "correlationID": "321e4567-e89b-12d3-a456-531531",
          "timeRequested": "2024-11-10T11:43:20.135Z",
          "timeEnded": "2024-11-10T11:43:25.135Z",
          "output": true
        }
    ]
  },
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

subscribeevent

Request

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "c370da58-69ae-4e83-bb5a-ac6cfb2fed54",
  "messageType": "request",
  "operation": "subscribeevent",
  "name": "overheated",
  "lastEventID": "0b69db8d-9b5a-4801-be76-8eb8f4af0d4a",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Response

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "response",
  "operation": "subscribeevent",
  "name": "overheated",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

Notification

{
  "thingID": "https://mythingserver.com/things/mylamp1",
  "messageID": "79057736-3e0e-4dc3-b139-a33051901ee2",
  "messageType": "notification",
  "operation": "subscribeevent",
  "name": "overheated",
  "data": 90,
  "timestamp": "2024-01-13T23:20:50.52Z",
  "correlationID": "5afb752f-8be0-4a3c-8108-1327a6009cbd"
}

I don't have a complete set of examples yet and there are still some questions that I think would need answering, but I wanted to see what people thought.

---

Notes:

1. In this proposal there would be no separate error message type, since the error would be contained in a response.
2. The ping and pong messages from the strawman proposal also don't fit neatly into this approach, though I'm not completely sure they're needed.
3. I'm not sure about how correlationIDs should work in queryaction and queryallactions messages, but I will file a separate issue for that.
4. I was too lazy to generate unique UUIDs for every message and match up correlationIDs, but hopefully you get the idea.

[benfrancis]

@hspaay @RobWin What do you think of this proposal?

[hspaay]

Ooh sorry, I must have overlooking this somehow.

In short: "I love it!" This addresses almost all issues I ran into in my experiments with strawman. It also easily supports expansion of operarations if there is such a need.

A few minor thoughts (not a big deal):

1. Should WriteProperty should return a status in its response. I've seen devices where writing a property can take up to 5 seconds or more. (1-wire nodes), so I treat them the same as actions. Another use-case for this is writing properties on devices that are asleep. A gateway could queue the request and return pending. What is your suggestion on solving this?
2. Are error results possible in all responses? I assume so, just want to confirm. Some requests might not be possible or allowed by the client.
3. Would you consider changing 'values' to 'output' in response to ReadXyz requests? It is the output to the request after all and it aligns nicely with the response to invokeAction and queryAction.

ps: I did eventually implement the strawman proposal in hiveot in addition to the second proposal I raised elsewhere. It is working but it was not a happy implementation and took 500 lines of golang extra. The above is so much better.

pps: Maybe the MQTT binding can adopt the same messaging. It too is an asysnc request/response protocol after all.

[benfrancis]

@hspaay wrote:

Should WriteProperty should return a status in its response. I've seen devices where writing a property can take up to 5 seconds or more. (1-wire nodes), so I treat them the same as actions. Another use-case for this is writing properties on devices that are asleep. A gateway could queue the request and return pending. What is your suggestion on solving this?

I don't think waiting 5 seconds for a response to a writeproperty operation is necessarily a problem over a WebSocket. In general though if an operation takes a very long period of time and you need to monitor its progress, I would suggest using an action rather than a property. I would rather not introduce the concept of the status of a property read/write since this doesn't exist in the WoT information model. At least with actions there is the concept of querying the status of an action, even if it is poorly defined. One option with sleeping devices could be to just respond without a value to acknowledge that the request has been received but not report that it has been acted upon (same as with write-only properties). A Consumer could then either read or observe the property if it wants to be updated when the change in value takes effect.

Are error results possible in all responses?

Yes.

Would you consider changing 'values' to 'output' in response to ReadXyz requests? It is the output to the request after all and it aligns nicely with the response to invokeAction and queryAction.

Personally I prefer a "name" and "value" pair for properties, where a "value" can be both an input to the operation (in the case of writing a property) and an output from the operation (in the case of reading a property). The differentiation from actions (and events) is intentional, because the terms "input" and "output" have special meanings in actions with their own dedicated data schemas in the Thing Description.

[hspaay]

Thank you for elaborating Ben

[hspaay]

I responded to action messages in #43. In short I think an ActionStatus object is needed based on my experience in hiveot. More details in the linked issue.

[hspaay]

On un unrelated note, it is curious to see that this proposal has the number 42