Streamdeck Plus

[c0nnex]

Recently got my Streamdeck Plus (ProductID 0x0084)
It features 8 normal buttons , 4 ecoders with images (imagesize unknown yet, but rectangle not square)

I'll play with it the next days and capture some reports.

UsbDeview report
report.zip

[c0nnex]

Normal Buttons: JPEG, Size 100x100 (Same palyoad paket as for XL)
Encoders: JPEG, Size 100x200 (Payload Paket Header is 16 Bytes)
Will need some rewrite of the internal code to reflect diffrent x/y dimensions. Might come along with a PR on the weekend

SideNote: I switched to MAGICK.net for all the image composition/resizing etc to bypass windows System.Drawing thread-limitations, but I'll do the PR for Graphics-implementation

[c0nnex]

Got the Streamdeck Plus implemented.
But it's some fiddling, because i did not want to touch OpenMacroBoard.SDK
I'll publish my branch in my Fork later today, and you can have a look if you want to do it that way (and i'llopen a pull request), or want adopt the OpenMacroBoard.SDK
Generally the currently layout of device data handling is not very flexible and i would suggest to redesign it.

[c0nnex]

OK after working deeper into it, it turns out that what i want to do with the plus ( handle encoder-image as one big image, like the SD does) would result in a major rewrite of all handling, so i skipped the OpenMacroBoard approach.
However, if anyone, is interested I'll push my "4x 200x100"-Images OMB-Implementation after code clean up, but it's based on an old branch (non-ImageSharp)

For anyone also working on this, here's the usb payload info:

Input-PayLoad:

           /* byte ReportId - 0x01
             * byte ReportType - 0x00 ButtonReport , 0x03 EncoderReport
             * ushort ReportSize - Number of state bytes following.            
             * <ReportSize> bytes with states. For Encoder Report first byte is Encoder-Turn flag
             * <Padding to ReportLength>
             */

If is EncoderReport AND first DataByte is 0x01:
following Databytes are signed bytes (- CountClockWise + Clockwiset turns, Abs(0 .. 16) , giving the speed of turning)
Else databytes are just 0/1 for release/press of the respective encoder (offset - 1 )

Output-Payload:
All Images for SD Plus (buttons and encoder) are jpeg encoded , not flipped.
Output report for the 8 buttons : identical to Streamdeck XL but 120x120 pixels

Output report for the encoder image (800x100 in total):
This is a "image difference rectangle report"
HeaderSize 16

/* 
0x02 // id
0x0c // Image Rectangle
ushort Left
ushort Top
uShort width
ushort height
byte isFinalPayLoad (0 / 1 )
ushort payLoadIndex (0 ... x)
ushort imagePayloadLength
byte padbyte (0)
<payload>
<padding to 1024>
*/
                var leftPixel = Encoders.KeyWidth * encoderId; // 200 * x ... Elgato itself handles the whole pic as one 800x100 picture and just sends diffrence rectangles. Pretty nice and fast.
                // Report ID
                data[0] = 2; 
                // Image-Rectangle Jpeg
                data[1] = 0x0c; 

                //Left
                data[2] = (byte)(leftPixel & 0xff);
                data[3] = (byte)(leftPixel >> 8);
                // Top
                data[4] = 0;
                data[5] = 0;
                // Width
                data[6] = (byte)(Encoders.KeyWidth & 0xff); // defaults to 200
                data[7] = (byte)(Encoders.KeyWidth >> 8);
                // Height
                data[8] = (byte)(Encoders.KeyHeight & 0xff); // defaults to 100
                data[9] = (byte)(Encoders.KeyHeight >> 8);

                data[10] = (byte)(isLast ? 1 : 0);

                data[11] = (byte)(pageNumber & 0xff);
                data[12] = (byte)(pageNumber >> 8);

                data[13] = (byte)(payloadLength & 255);
                data[14] = (byte)(payloadLength >> 8);

                // Padding
                data[15] = 0;

[Mutex666]

Hi @c0nnex , thanks for your job on the Stream Deck Plus. I just received mine and I would be curious to dig into your code to see how things goes. I looked to implement it myself in the Streamdecksharp code but hardware dialog is quite new for me so I'm struggling a little to see how I can get report of the HID device and so on.

[Necromantic]

Thanks to @c0nnex for his ground work.

I've just implemented this on my side. Had to mess with IMacroBoard, add more event/delegate types, additional hardware information and a few method overloads with different information. It didn't take all that much work, but it's still a bit messy. As was said before this might need to be redesigned to be a bit more dynamic.
I've implemented the touchscreen output with a rectangle (roi) approach independent of the encoders, so I can draw images over any area of any size I want on the screen.

For the additional input reports the information is as follows (reduced to the important data).
Byte 2 (ReportType): 0x00 ButtonReport, 0x02 TouchReport, 0x03 EncoderReport

TouchReport:
Byte 5 (EventType): 0x01 TapEvent, 0x02 HoldEvent, 0x03 SwipeEvent

TapEvent:
Byte 7 & 8: Touch X-Position in pixels (unsigned short)
Byte 9 & 10: Touch Y-Position in pixels (unsigned short)

HoldEvent:
Byte 7 & 8: Touch X-Position in pixels (unsigned short)
Byte 9 & 10: Touch Y-Position in pixels (unsigned short)

SwipeEvent:
Byte 7 & 8: Start X-Position in pixels (unsigned short)
Byte 9 & 10: Start Y-Position in pixels (unsigned short)
Byte 11 & 12: End X-Position in pixels (unsigned short)
Byte 13 & 14: End Y-Position in pixels (unsigned short)

[Mutex666]

Wow , fantastic news @Necromantic. Thanks for your work on the subject. Can't wait to test this :-)

[jpk6789]

Can't wait to implement this in our software. Would be nice to use the encoders as another input besides the buttons of the other stream decks. Are there any updates on the implementation? :)

[wischi-chr]

I'm currently planning a redesign of the interfaces in OpenMacroBoard, because I also want to support devices like Streamdeck Plus and Loupedeck CT which looks like this:

[flat-eric147]

I'm currently planning a redesign of the interfaces in OpenMacroBoard, because I also want to support devices like Streamdeck Plus and Loupedeck CT which looks like this:

I would love to see you implementing the Loupedeck. I used their plugin api and it's quite limiting. Having some low level control on the Loupedeck would be a big plus for me. Of course I would also love to see the StreamDeck + implemented here!

[Necromantic]

I second that. A redesign for future "macro boards" seems to be needed.
I've used my hacky implementation of the Stream Deck Plus to control an interactive media installation from a Raspberry Pi.
I might open up a fork with that implementation but because it's so hacky I've refrained from it so far.

I'm currently looking at the Stream Deck Plus vs. the Loupedeck Live/CT for various future projects. Currently I'm leaning towards the Stream Deck Plus because of this repository and the possibilities it gives me. But if this would expand to more generic systems with these new inputs/outputs (Touch Displays, Encoders etc.) this would open up many more options

@wischi-chr Has there been any progress/effort from your part towards this goal or is that just a future plan?
Have you put any thought into the abstractions of the redesign?

[wischi-chr]

Sadly no, there hasn't been much progress since. I still want to implement this but I don't want to make any promises when I will find time to do that.

I'm currently not sure what I want the interface to look like from a developers perspective that uses the library.

I'd like it to be intuitive an yet strictly typed but also reduce the number of different types needed. For example I plan to represent the color LED buttons also as "BitmapKeys" but with single 1x1 pixel "bitmap". This way all the same method and factories to create and clear keys can still be used exactly like the other keys.

I recently looked into how some operating systems designed their interfaces for gamepads, but the situation is a bit different because most gamepads don't have visual feedback/outputs like displays or LEDs.

[wojciechsura]

Do you plan the library to be cross-platform? If not, I would drop SixLabors library for images and just stick with GDI+ Bitmap - it is more than sufficient to handle everything needed for the *decks.

I'd be also glad to see LoupeDeck implementation - the device is really cool, but unfortunately LoupeDeck's software lags behind their fantastic hardware. I could also add support for LoupeDeck in my OpenMacroKeyboard project.

I can help somewhat in the implementation, but I'm not that good in hardware RE, so I'd probably need some guidance on how to do that.

[Necromantic]

I have it running on a Raspberry Pi for a project. So the cross-platform libraries do have a use.

[wischi-chr]

@wojciechsura it actually already is cross-platform and that's the reason why it depends on ImageSharp (because the library internally needs to encode jpegs and resize images, etc.) - I'm using it on windows and linux (Ubuntu) myself.

I have a LoupeDeck already in my drawer but haven't found the time to implement it. The reverse engineering part isn't actually the most difficult part.

I have pretty strong opinions about what the code and the API should look like and would a "comfortable" API would be. I thought about how to "generalize" the API for devices with rotary engoder knobs, sliders, Display-Buttons (like the Stream Deck) but also allow users to handle touch events (the LoupeDeck buttons is actually a touch screen), etc. and all that without breaking (at least not completely changing) the way current users work with their "simple" macro boards.

It's still "on my list" but sadly it's a pretty long list and I don't want to make more promises I can't keep.