Conversation
| // Attempt automatic error recovery to clear any previous error state | ||
| // This ensures the robot is ready for control even if the previous run ended with an error | ||
| try { | ||
| robot->automaticErrorRecovery(); |
There was a problem hiding this comment.
Please remove redundant comments -> I guess the function 'automaticErrorRecovery' explains already what the comment is telling :)
common/src/robot_api.cpp
Outdated
| robot = std::make_unique<franka::Robot>(robotIPString); | ||
|
|
||
| // Attempt automatic error recovery to clear any previous error state | ||
| // This ensures the robot is ready for control even if the previous run ended with an error |
There was a problem hiding this comment.
Why is all of this necessary?
Wasn't the user able to build the whole thing so far?
There was a problem hiding this comment.
The user can still build the project at will by choosing freely any desired build method. These are definitely not a "must".
The story so far regarding this build infra:
These are dev tools developed so we can handle testing and evaluation complexity as the toolbox involves binaries for the matlab/simulink mex front end for Windows and Ubuntu Host PC, combined with the back-end layer (robot_api stuff under common + rpc server) for both Jetson (Franka AI companion) + amd64 targets.
This build infrastructure allow us to drive the github workflows for delivering a standalone "drag-and-drop" final franka.mtlbx which is the standard Matlab Toolbox distribution format.
There was a problem hiding this comment.
Same answer as previous comment.
|
|
||
| # Build Cap'n Proto (static libraries with PIC) | ||
| CAPNP_VERSION="1.0.2" | ||
| curl -O https://capnproto.org/capnproto-c++-${CAPNP_VERSION}.tar.gz |
There was a problem hiding this comment.
We opted for a client-server architecture in order to add support for the Franka AI companion (and generally jetson platforms) for our Matlab API (not simulink in this case). Main drive for the capnproto against other alternatives was it's light weight structure, quite simple straightforward and clean API in combination with its delivery of a simple out-of-the-box rpc server framework.
There was a problem hiding this comment.
I don't understand it. It doesn't make sense to add support via proto towards the matlab api. Either you can use the matlab api directly e.g. via simulink or matlab or you don't need matlab and go directly via libfranka :)
If we would add support to communicate via protos (or something similar), we would introduce that on a libfranka level.
There was a problem hiding this comment.
The problem formulation and the motivation behind this was at the time (~1.5-2y ago):
- Windows support if very attractive for Matlab users. Windows has no rt capability (at least adequate for robustly maintaining the FCI 1khz thing). Additionally RT for some workstation setup is painful even for Ubuntu (NVIDIA GPU confict etc).
- We do want some high-level Matlab API support for the robot (not libfranka 1khz level rather than just simple joint point to point motion, traj open-loop follow up) as a feature in order for the users to be able to rapid-prototype and evaluate Matlab infra directly on the robot.
- Support for the Franka AI companion in conjunction Matlab (client running in workstation PC non rt)
Given 1, 2 and 3 we opted for a server-client architecture for supporting our Matlab API.
This is very similar actually with other popular high-level APIs for the robot in python like polymetis and deoxys control.
This architecture was not introduced with this PR btw and is already active since the release of version 1.0.0.
If things have changed regarding the 1 2 and 3 constraints we can drop support for the high-level Matlab API all together. Or supported only in the case where the Matlab is running in RT kernel Ubuntu Host PC.
Please let me know on how you'd like to proceed on that. In my humble opinion this discussion is outside the scope of the current PR and should be part of the overall project dev roadmap.
There was a problem hiding this comment.
Yes of course, that topic is outside of the scope of this :)
I was mostly curious on the 'why'.
|
|
||
| // Set Joint Impedance | ||
| if (!strcmp("set_joint_impedance", cmd)) { | ||
| if (nlhs != 1 || nrhs != 3) |
There was a problem hiding this comment.
These are the number of the given return outputs (nlhs) and and the given function arguments (nrhs) that the user has typed in matlab when calling a function. (standard Matlab nomenclature).
Like e.g >> [a,b,c] = matlab_foo(d,e,f,g) would mean nlhs = 3 and nrhs = 4.
Hence some sanity check in this case when the user is calling our Matlab API methods.
| } | ||
|
|
||
| // Set Joint Impedance | ||
| if (!strcmp("set_joint_impedance", cmd)) { |
There was a problem hiding this comment.
Why do you need to use a strcmp and not just ==?
There was a problem hiding this comment.
It's a balance act dealing with the Matlab mex function style which is traditionally C :).
(Yes there is a more recent C++ infra for Matlab mex function but I've faced interoperability issues in the past and still the standard C mex api is the more mature and "standard" framework.)
Based on that e.g for parsing the matlab cmd we do:
char cmd[64];
mxGetString(prhs[0], cmd, sizeof(cmd)))
which "forces us" into the C-style strcmp().
Again "matlaby" mex function idiomatic stuff.
| } | ||
|
|
||
| // Set Joint Impedance | ||
| if (!strcmp("set_joint_impedance", cmd)) { |
There was a problem hiding this comment.
Where these functions not yet available or why the introduction?
There was a problem hiding this comment.
Yep they were not available as part of the Matlab API of ours. The previous version 1.0.0 included a subset of the libfranka available methods. This PR includes these residual functions following the commonly agreed development roadmap of the project.
| #include <franka/robot.h> | ||
| #include <array> | ||
|
|
||
| kj::Promise<void> FrankaRobotRPCServiceImpl::setK( |
There was a problem hiding this comment.
We opted for a client-server architecture in order to add support for the Franka AI companion (and generally jetson platforms) for our Matlab API (not simulink in this case). Main drive for the capnproto against other alternatives was it's light weight structure, quite simple straightforward and clean API in combination with its delivery of a simple out-of-the-box rpc server framework.
| '-lkj-async', ... | ||
| '-lWs2_32', ... | ||
| 'CXXFLAGS="\$CXXFLAGS -std=c++14"', ... | ||
| 'COMPFLAGS="$COMPFLAGS /std:c++17"', ... |
There was a problem hiding this comment.
Why did you change this? It is leading to problems
There was a problem hiding this comment.
This is the Matlab API mex build for Windows Host PC. Switching the syntax and stepping up to C++17 was necessary for ensuring proper build.
Could you elaborate on what sort of problems exactly you run into so I can provide a valid resolution?
There was a problem hiding this comment.
You also changed it for linux.
I took the toolbox built by default and just trying to run the first simulink example I get:
Starting build procedure for: cartesian_impedance_control
gcc -c -fwrapv -fPIC -O3 -fno-loop-optimize -fno-aggressive-loop-optimizations -msse2 -DCLASSIC_INTERFACE=0 -DALLOCATIONFCN=0 -DEXT_MODE=1 -DMAT_FILE=1 -DONESTEPFCN=1 -DTERMFCN=1 -DMULTI_INSTANCE_CODE=0 -DINTEGER_CODE=0 -DMT=0 -DON_TARGET_WAIT_FOR_START=1 -DTID01EQ=0 -DMODEL=cartesian_impedance_control -DNUMST=1 -DNCSTATES=0 -DHAVESTDIO -DRT -DUSE_RTMODEL -I/home/gome_gu/FM_tests -I/home/gome_gu/FM_tests/cartesian_impedance_control_grt_rtw -I/usr/local/MATLAB/R2023a/extern/include -I/usr/local/MATLAB/R2023a/simulink/include -I/usr/local/MATLAB/R2023a/rtw/c/src -I/usr/local/MATLAB/R2023a/rtw/c/src/ext_mode/common -I/usr/local/MATLAB/R2023a/toolbox/coder/rtiostream/src -I/usr/local/MATLAB/R2023a/toolbox/coder/rtiostream/src/rtiostreamtcpip -I/usr/local/MATLAB/R2023a/toolbox/coder/rtiostream/src/utils -I/home/gome_gu/MATLAB\ Add-Ons/Toolboxes/Franka\ Toolbox\ for\ MATLAB/common/include -I/home/gome_gu/MATLAB\ Add-Ons/Toolboxes/Franka\ Toolbox\ for\ MATLAB/libfranka/include -I/home/gome_gu/MATLAB\ Add-Ons/Toolboxes/Franka\ Toolbox\ for\ MATLAB/libfranka/common/include -o "rt_logging.o" "/usr/local/MATLAB/R2023a/rtw/c/src/rt_logging.c"
/usr/local/MATLAB/R2023a/rtw/c/src/rt_logging.c: In function ‘rt_FixupLogVar’:
/usr/local/MATLAB/R2023a/rtw/c/src/rt_logging.c:1296:19: warning: ignoring return value of ‘fread’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
1296 | (void)fread(var->data.re, elSize, nEl, fptr);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
g++ -c -fwrapv -fPIC -O3 -fno-loop-optimize -fno-aggressive-loop-optimizations -msse2 -DCLASSIC_INTERFACE=0 -DALLOCATIONFCN=0 -DEXT_MODE=1 -DMAT_FILE=1 -DONESTEPFCN=1 -DTERMFCN=1 -DMULTI_INSTANCE_CODE=0 -DINTEGER_CODE=0 -DMT=0 -DON_TARGET_WAIT_FOR_START=1 -DTID01EQ=0 -DMODEL=cartesian_impedance_control -DNUMST=1 -DNCSTATES=0 -DHAVESTDIO -DRT -DUSE_RTMODEL -I/home/gome_gu/FM_tests -I/home/gome_gu/FM_tests/cartesian_impedance_control_grt_rtw -I/usr/local/MATLAB/R2023a/extern/include -I/usr/local/MATLAB/R2023a/simulink/include -I/usr/local/MATLAB/R2023a/rtw/c/src -I/usr/local/MATLAB/R2023a/rtw/c/src/ext_mode/common -I/usr/local/MATLAB/R2023a/toolbox/coder/rtiostream/src -I/usr/local/MATLAB/R2023a/toolbox/coder/rtiostream/src/rtiostreamtcpip -I/usr/local/MATLAB/R2023a/toolbox/coder/rtiostream/src/utils -I/home/gome_gu/MATLAB\ Add-Ons/Toolboxes/Franka\ Toolbox\ for\ MATLAB/common/include -I/home/gome_gu/MATLAB\ Add-Ons/Toolboxes/Franka\ Toolbox\ for\ MATLAB/libfranka/include -I/home/gome_gu/MATLAB\ Add-Ons/Toolboxes/Franka\ Toolbox\ for\ MATLAB/libfranka/common/include -o "cartesian_impedance_control.o" "/home/gome_gu/FM_tests/cartesian_impedance_control_grt_rtw/cartesian_impedance_control.cpp"
In file included from /home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h:14,
from /home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/common/include/robot_api.h:19,
from /home/gome_gu/FM_tests/cartesian_impedance_control_grt_rtw/cartesian_impedance_control.h:27,
from /home/gome_gu/FM_tests/cartesian_impedance_control_grt_rtw/cartesian_impedance_control.cpp:17:
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:242:24: error: ‘optional’ in namespace ‘std’ does not name a template type
242 | const std::optional &maximum_velocity = std::nullopt)
| ^~~~~~~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:242:19: note: ‘std::optional’ is only available from C++17 onwards
242 | const std::optional &maximum_velocity = std::nullopt)
| ^~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:242:32: error: expected ‘,’ or ‘...’ before ‘ &maximum_velocity = std::nullopt)
| ^
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:242:13: error: default argument missing for parameter 6 of ‘research_interface::robot::Move::Request::Request(research_interface::robot::Move::ControllerMode, research_interface::robot::Move::MotionGeneratorMode, const research_interface::robot::Move::Deviation&, const research_interface::robot::Move::Deviation&, bool, int)’
242 | const std::optional &maximum_velocity = std::nullopt)
| ^~~~~~~~~~~~~~~~~~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:241:18: note: ...following parameter 5 which has a default argument
241 | bool use_async_motion_generator = false,
| ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h: In constructor ‘research_interface::robot::Move::Request::Request(research_interface::robot::Move::ControllerMode, research_interface::robot::Move::MotionGeneratorMode, const research_interface::robot::Move::Deviation&, const research_interface::robot::Move::Deviation&, bool, int)’:
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:248:58: error: ‘struct std::array<double, 7>’ has no member named ‘has_value’
248 | if (use_async_motion_generator && maximum_velocity.has_value()) {
| ^~~~~~~~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:249:61: error: base operand of ‘->’ has non-pointer type ‘std::array<double, 7>’
249 | if (this->maximum_velocity.size() > maximum_velocity->size()) {
| ^~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:254:29: error: base operand of ‘->’ has non-pointer type ‘std::array<double, 7>’
254 | maximum_velocity->cbegin(), maximum_velocity->cend(), this->maximum_velocity.begin());
| ^~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/common/include/research_interface/robot/service_types.h:254:57: error: base operand of ‘->’ has non-pointer type ‘std::array<double, 7>’
254 | maximum_velocity->cbegin(), maximum_velocity->cend(), this->maximum_velocity.begin());
| ^~
In file included from /home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/common/include/robot_api.h:19,
from /home/gome_gu/FM_tests/cartesian_impedance_control_grt_rtw/cartesian_impedance_control.h:27,
from /home/gome_gu/FM_tests/cartesian_impedance_control_grt_rtw/cartesian_impedance_control.cpp:17:
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h: At global scope:
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h:718:18: error: ‘optional’ in namespace ‘std’ does not name a template type
718 | const std::optional& maximum_velocities);
| ^~~~~~~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h:718:13: note: ‘std::optional’ is only available from C++17 onwards
718 | const std::optional& maximum_velocities);
| ^~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h:718:26: error: expected ‘,’ or ‘...’ before ‘& maximum_velocities);
| ^
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h:859:37: error: ‘optional’ in namespace ‘std’ does not name a template type
859 | const std::optional& maximum_velocities)
| ^~~~~~~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h:859:32: note: ‘std::optional’ is only available from C++17 onwards
859 | const std::optional& maximum_velocities)
| ^~~
/home/gome_gu/MATLAB Add-Ons/Toolboxes/Franka Toolbox for MATLAB/libfranka/include/franka/robot.h:859:45: error: expected ‘,’ or ‘...’ before ‘& maximum_velocities)
| ^
gmake: *** [cartesian_impedance_control.o] Error 1
Build procedure for cartesian_impedance_control aborted due to an error.
There was a problem hiding this comment.
"libfranka/include/franka/robot.h:859:32: note: ‘std::optional’ is only available from C++17 onwards" that's not related to our Matlab client backend and the mex binaries build. It's a requirement that from latest libfranka 0.20.5 requiring C++ 17 or newer.
Can you attempt the following simulink model settings before building?

There was a problem hiding this comment.
I cannot select C++17 there, either 11 or 3
There was a problem hiding this comment.
Can we not use 2023? I would like to keep the version. Default is 11 in my setup yes.
There was a problem hiding this comment.
From: https://www.mathworks.com/help/pdf_doc/rtw/rn.pdf
So my mistake minimum Matlab version for compatibility with latest libfranka is 2024b and not 2024a!
There was a problem hiding this comment.
I am really confused about this. What did we change in libfranka so that we need to update this? I saw that we used std::optional already in the past and, for my surprise, we never had a problem
There was a problem hiding this comment.
It's pretty clear based on the compiler error. We need to enforce the -std=c++17 option to the compiler :). In my case I'm doing dev in Ubuntu 24.04 and gcc 13 which defaults to c++17. In your case if you operate in Ubuntu 22.04 you'll need to fix that either though a newer Matlab version and choosing c++17 standard in simulink model settings or alternatively make your gcc system compiler auto defaulting to c++17 (upgrading to gcc 13 potentially).
Matlab 2024b would be the quickest way to get there now I reckon.
There was a problem hiding this comment.
"libfranka/include/franka/robot.h:859:32: note: ‘std::optional’ is only available from C++17 onwards" that's not related to our Matlab client backend and the mex binaries build. It's a requirement that from latest libfranka 0.20.5 requiring C++ 17 or newer.
Can you attempt the following simulink model settings before building?
Could you add this somewhere in the docu?
| - **Simulink Coder Support** — Generate and deploy C++ code to real-time Linux targets | ||
| - **Cross-Platform** — Works on Windows and Linux, targets x86_64 and ARM64 (Jetson) | ||
|
|
||
| 2. Set the desired libfranka version in the `libfranka_ver.csv` file under the `configs` folder |
There was a problem hiding this comment.
This part is missing in the new additions to the docu. The user should know how to change the libfranka version.
Also, now is the binaries are built in docker right? Is this function 'franka_toolbox_binaries_all_build();' still useful?
There was a problem hiding this comment.
I've added analytical instructions in new docs page "custom_build" exactly for that reason.
franka_toolbox_binaries_all_build(); is now a wrapper around the dockerized build so yep still useful.
There was a problem hiding this comment.
In the custom build section, there is no reference on how to change the libfranka version. Could you add it?
There was a problem hiding this comment.
I get this warnings during the toolbox installation (I followed the steps from the docu):
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/franka_simulink_library/bin.zip
In franka_toolbox_install/tryUnzip (line 65)
In franka_toolbox_install/unpackBinaries (line 38)
In franka_toolbox_install (line 20)
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/common/bin.zip
In franka_toolbox_install/tryUnzip (line 65)
In franka_toolbox_install/unpackBinaries (line 42)
In franka_toolbox_install (line 20)
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/common/bin_arm.zip
In franka_toolbox_install/tryUnzip (line 65)
In franka_toolbox_install/unpackBinaries (line 44)
In franka_toolbox_install (line 20)
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/franka_robot_server/bin.tar.gz
In franka_toolbox_install/tryUntar (line 78)
In franka_toolbox_install/unpackBinaries (line 49)
In franka_toolbox_install (line 20)
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/franka_robot_server/bin_arm.tar.gz
In franka_toolbox_install/tryUntar (line 78)
In franka_toolbox_install/unpackBinaries (line 50)
In franka_toolbox_install (line 20)
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/franka_robot/bin.zip
In franka_toolbox_install/tryUnzip (line 65)
In franka_toolbox_install/unpackBinaries (line 53)
In franka_toolbox_install (line 20)
Warning: Name is nonexistent or not a directory: /home/gome_gu/franka_matlab/franka_robot/bin
In path (line 109)
In addpath (line 86)
In franka_toolbox_install/unpackBinaries (line 54)
In franka_toolbox_install (line 20)
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/dependencies/libfranka.zip
In franka_toolbox_install/tryUnzip (line 65)
In franka_toolbox_install/unpackBinaries (line 58)
In franka_toolbox_install (line 20)
Warning: Archive not found, skipping: /home/gome_gu/franka_matlab/dependencies/libfranka_arm.zip
In franka_toolbox_install/tryUnzip (line 65)
In franka_toolbox_install/unpackBinaries (line 59)
In franka_toolbox_install (line 20)
Franka Toolbox installation completed successfully!
This leads to the following error in the simulink examples:
Error in S-function 'cartesian_impedance_control/Cartesian Impedance Controller/Duration Period': S-Function 'get_duration_period' does not exist
Component:Simulink | Category:Model error
Error in S-function 'cartesian_impedance_control/Robot State': S-Function 'get_robot_state' does not exist
Component:Simulink | Category:Model error
Error in S-function 'cartesian_impedance_control/Robot State1': S-Function 'get_robot_state' does not exist
There was a problem hiding this comment.
We do not include binaries in source code. If you want to work from source code directly after cloning you'd need to do
./docker/build.sh --libfranka #optionally amd64 or arm64
franka_robot_mex();
franka_simulink_library_mex();
as described in custom build section in new docs.
and then
franka_toolbox_install();
I've improved user prompting during the install operation for a cleaner workflow.
There was a problem hiding this comment.
Missing required archives:
- franka_simulink_library/bin.zip
- common/bin.zip
- franka_robot_server/bin.tar.gz
- franka_robot/bin.zip
- dependencies/libfranka.zip
There was a problem hiding this comment.
Missing required archives:
- franka_simulink_library/bin.zip
- common/bin.zip
- franka_robot_server/bin.tar.gz
- franka_robot/bin.zip
- dependencies/libfranka.zip
Not even a single binary archive has been found! Something's off with the build process. I've improved the ./build.sh process for double checking in case of potential failure.
Please do
./docker/build.sh --libfranka 0.20.5>>franka_robot_mex();
>>franka_simulink_library_mex();
The build.sh should now throw in case something's off in the end. Also please check exactly the outputs during the mex builds...
There was a problem hiding this comment.
Could it be that the dist_make is removing the binaries?
There was a problem hiding this comment.
yep that's it. I thought you were working from the cloned repo directly doing franka_toolbox_install(); there ... I've improved the docs instructions and the default behavior for dist_make is now to preserve the local artifacts.
There was a problem hiding this comment.
Yes, that was it. Now it works with matlab2025b
|
|
||
| Default values: ``[3000, 3000, 3000, 300, 300, 300]`` | ||
|
|
||
| Guiding Mode |
There was a problem hiding this comment.
Why do we need this mode? I tried but nothing change after setting all to true. Can we delete it?

New Franka Toolbox release 4.0.0 including:
FrankaRobotSettingsclass for centralized robot configuration.setJointImpedance,setCartesianImpedance), guiding mode (setGuidingMode), frame transformations (setEE,setK), and motion control (stop).Full build with github workflow build-and-release.yml which is activated when pushing tags to the remote.
you can check the outcome of the release process here --> https://github.com/kvasios/franka_toolbox_for_matlab/releases