Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev-threads Qt example fork changes #37

Open
wants to merge 86 commits into
base: master
Choose a base branch
from

Conversation

ClaudioHoffmann
Copy link

For our Solum integration, we've decided to just extend the Qt example application. The big feature of our fork will be OpenIGTLink support, but we're also going to add more generally useful changes such as increased feedback and user-friendliness for the GUI. That's why it makes sense to integrate them back into the official repository, and it will also simplify upgrades to new API versions for everyone.
I'm going to push all changes to https://github.com/dev-threads/solum/tree/master as they get developed and describe them in new comments to this PR until it gets closed, so feel free to cherry-pick just the features you want, if any.

The first feature replaces the single-line status bar with a multi-line textbox. Some API and GUI actions result in multiple lines of status text being shown in quick succession, but the current status bar can only show the last one of them. This makes it more difficult than necessary to debug issues, especially during connection to the probe where the generic "connection failed" message previously overwrote the more helpful ConnectionError message. A multiline textbox helps massively there, and can also double nicely as a log.

Solum logging

@ClaudioHoffmann
Copy link
Author

Added two smaller bug fixes:

  • Removed a second call to solumDestroy() that led to use-after-free crashes when debugging the app,
  • and worked around a label color bug with QPushButton in the default Qt Windows style by using the Fusion style instead, which is bundled with Qt as well:

Solum Fusion

@julien-l
Copy link
Contributor

julien-l commented Jul 5, 2023

Thanks we will take look!

@ClaudioHoffmann
Copy link
Author

Great! Fixed two more dangling pointer errors during shutdown that caused crashes in debug mode in the latest commits, and prepared improved UI feedback for long-running actions.

@ClaudioHoffmann
Copy link
Author

Redesigned the connection-related parts of the UI in a more self-documenting way, making them more straightforward and less error-prone to use:

  • The progress bar is now hidden until the first actual progress event.
  • The program now automatically starts a BLE device search on startup.
  • Since all interesting features of a TCP connection require a downloaded certificate, I moved the previous contents of the Cloud tab to the TCP tab. I also changed all TCP-related UI to only be enabled after a certificate was downloaded.
  • The functional difference between the previous Wi-Fi and AP connection options is now briefly explained directly within the BLE tab. All of these options now also remain disabled until a BLE connection has been established.
  • The SSID and password fields are now properly labeled. Previously, their purpose was only described in their tooltips.
  • The Probe model combo box now only receives the subset of models that are certified to be used with the Cloud API token. solumLoadApplication() silently does nothing when providing a model string that does not match the connected probe, so a limited subset can reduce another potential user error. This could be even nicer if we remembered the Bluetooth-provided serial number of the TCP-connected probe and used it to automatically set the correct model, which would allow us to remove the combo box altogether, especially for organizations that can use more than one scanner. But that's a feature for another day.
  • Finally, I added more status box updates to provide additional reactivity. (And joy while using the program.)

This is probably the final set of commits before I add OpenIGTLink support.

BLE tab of the dev-threads Solum Qt example UI redesign
TCP tab of the dev-threads Solum Qt example UI redesign

@ClaudioHoffmann
Copy link
Author

Alright, added another bunch of UI redesigns after all, together with complete support for streaming the received images via the OpenIGTLink protocol. Looks like this is all we planned to do with Solum.
I've cleanly separated the UX-related commits from the OpenIGTLink support, so if you don't want to keep the latter, you can simply remove all commits after and including the git subrepo clone one.

Solum Final OpenIGTLink

An overview of the final set of improvements:

  • I've tried to repair the currently broken Solum API include files and their inclusions. After manually specifying the path to solum.lib, the Qt GUI example now compiles and runs again.

  • The TCP connection is now coupled to the BLE one. The IP and port are announced via BLE, can randomly change between connection attempts, and aren't valid before the message is published anyway, so it makes little to use editable input fields here.

    Solum Final TCP unconnected

  • This coupling also allows the probe model to be automatically derived from the serial number of the BLE-connected probe, by taking the model string from the certification data sent by the cloud API. It's still checked against the list of Solum-supported probes, returned by solumProbes().

    Solum Final TCP automatic probe model

  • I've combined the previously distinct Load and Run steps into a single button. While this adds the 1-second workaround delay for solumLoadApplication() lack of a callback to the Run step as well, having to only click one button instead of two more than makes up for it in usability. Especially since it's unclear if the loaded application is retained after losing connection to a probe and then reconnecting.

    Solum Final TCP received IP and port

  • I added a new Certificates tab with an overview of all downloaded certificates and their effective and expiry dates.

    Solum Final Certificates

  • All downloaded certificates are now persisted in the QSettings file. This allows the app to be used in a more offline setting…

  • …and without requiring the certificate to be set manually. I've removed the respective button, as it's strictly worse than this automatic solution.

  • I've reordered all widgets on the Params tab, adding labels to the dials and a group box to the TGC sliders.

    Solum Final Params

@clariusk
Copy link
Contributor

Thanks for making the PR; we're about to release a pilot in early September, so I'll recommend you merge after our updates to the repo (I've actually done some more significant changes to the Qt Desktop program as well). Afterwards, I'll review your changes for merging back into main. Thanks again for the contributions!

@ClaudioHoffmann
Copy link
Author

Alright, will do!

@clariusk
Copy link
Contributor

@ClaudioHoffmann we've made some of the more major changes to the UI with version 11.0, if you wanted to revisit your PR and update as necessary, would be happy to review again, thanks!

This restores the layout of the v10.4.0 UI, since our fork will add
more certificate-related widgets.
Some operations result in multiple lines of text being shown in quick
succession, but a status bar shows only the last one of them. This
makes it more difficult to debug any issues that might occur,
especially during connection to the probe, where the generic
`"connection failed"` message previously overwrote the more helpful
`ConnectionError` message. A multiline textbox helps massively there,
and can also double nicely as a log.
A small bit of polish to differentiate it from a regular editable
textbox.
main() already contains such a call. Since solumInit() is also called
there, it's the only place where this call is appropriate. Calling this
function within Qt causes use-after-free errors in the main window
cleanup code.
…el colors

Kind of unfortunate, but the `fusion` style looks nicer anyway.
…dow class

Previously, the destructor for `Ble` instance of the main window class
ran after the deletion of `ui_`. This caused a rather involved
use-after-free bug if a probe is connected via Bluetooth, through the
following sequence of method calls:

• `Ble::~Ble()`
• `Ble::disconnectFromProbe()`
• `QLowEnergyController::disconnectFromDevice()`
• `QLowEnergyController::disconnected()` signal
• `QLowEnergyController::disconnected()` slot, connected in
  `Ble::connectToProbe()`
• `Ble::powerReady()` signal
• `Ble::powerReady()` slot, connected in `Solum::Solum()`

This last slot then tries to access fields of the deleted `ui_`
pointer.

`Ui::Solum` doesn't track any object lifetimes itself. It only tracks
raw pointers into the Qt widget tree, and doesn't even have a
destructor or a base class with a destructor. Therefore, we don't even
have to rely on C++'s well-defined destructor order here
(https://isocpp.org/wiki/faq/dtors#order-dtors-for-members), and
turning the instance from a manually `new`ed object into a plain class
member will just do the right thing.
Sure, it's a more sweeping change than heap-allocating `Ble` instead
and explicitly deleting it before `ui_`, but less manual lifetime
management is always preferable.
All higher states after `QLowEnergyController::ConnectedState` also
represent an existing connection.
Relying on all 4 prefix characters is better than relying on just 3 of
them.
ClaudioHoffmann and others added 13 commits November 17, 2023 05:07
We'd like to have a separate SUBDIRS project to combine the new
OpenIGTLink library project with the Qt example code.
The very unnecessary files generated by the original CMake scripts.
A build system where the supposed way to consume a library from an
application is so broken on Windows that it requires manual directory
specifications to get working… qmake might just be worse than CMake.

And yes, this codebase throws enough integer truncation warnings that
we just disable them all.
No disconnect detection yet, though. It seems as if disconnects can
only be detected by reading from the client and having that fail?
Also handling client reconnection, since this is where we have to
detect disconnects.
A simple first attempt, calculating FPS simply based on the time it
took between sending two images.
A small adjustment for less jumpy FPS numbers.
Setting the number of components makes the receiving node
a vector volume node. To have correct values in the received data,
the scalar type needed to be set to uint8.
@ClaudioHoffmann
Copy link
Author

Oh, I already updated it one month ago, and rebased our code on top of v11.0.0 to preserve any relevant changes from that version.

mcfly3001 and others added 15 commits April 17, 2024 10:07
Add some folders and files to gitignore which contain user configs and build files.
The file is not within the source dir, but one layer above.
Use QTs permission feature for bluetooth. Only if permissions are granted, the BLE search button will be enabled.
The Clarius probe returns a quoted string for the IP address. In order to connect, the quotes needed to be removed.
On macos, the settings.ini file could not be read from the app directory. Fix by using a writable location path.
Mac adjustments and update to latest SDK version 11.3.0
Disable all energy saving options to avoid the scanner going
to freeze mode during a demo.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants