|
| 1 | +/** @mainpage API Reference |
| 2 | + |
| 3 | +Introduction |
| 4 | +============ |
| 5 | + |
| 6 | +%libfreenect2 is an open source cross-platform driver for Kinect for Windows v2 |
| 7 | +devices. For information on installation and troubleshooting, see the |
| 8 | +[GitHub repository](https://github.com/OpenKinect/libfreenect2). |
| 9 | + |
| 10 | +This documentation is designed for application developers who want to extract |
| 11 | +and use depth and color images from Kinect v2 for further processing. |
| 12 | +Additional questions and comments not covered by this documentation can be |
| 13 | +posted to [GitHub issues](https://github.com/OpenKinect/libfreenect2/issues). |
| 14 | + |
| 15 | +This documentation may require some understanding on camera calibration and 3-D |
| 16 | +geometry. |
| 17 | + |
| 18 | +Features |
| 19 | +======== |
| 20 | + |
| 21 | +- Color image processing |
| 22 | +- IR and depth image processing |
| 23 | +- Registration of color and depth images |
| 24 | +- Multiple GPU and hardware acceleration implementations for image processing |
| 25 | + |
| 26 | +### Issues and Future Work |
| 27 | + |
| 28 | +- Audio. There is basic access to Kinect v2's audio via ALSA (Linux). However, |
| 29 | + this is directional audio with intricate calibration, which is probably |
| 30 | + beyond the scope of this image processing library. |
| 31 | +- Unstable USB and crashes. Due to differences in a vast range of hardware, it |
| 32 | + is very hard to test them all. Also, the libusb usage in %libfreenect2 may |
| 33 | + miss a lot of error checking and simply crash. This can be improved. |
| 34 | +- Firmware upload. This is being worked on. Use Windows for this right now. |
| 35 | +- Example of multiple Kinects. |
| 36 | +- Example utility of dumping image frames. |
| 37 | +- API for pausing, or on-demand processing. |
| 38 | +- Verification of systematic errors through accurate calibration. |
| 39 | +- Bindings for C, Python, Java, etc. |
| 40 | + |
| 41 | +Getting Started |
| 42 | +=============== |
| 43 | + |
| 44 | +To read the API documentation, start with the [Modules](modules.html) page |
| 45 | +which nicely organizes classes according to their functionalities. |
| 46 | + |
| 47 | +Example programs can be found in the source distribution under the `examples` |
| 48 | +directory. There also includes an example CMake build system for a standalone |
| 49 | +application that uses %libfreenect2 binary installation. |
| 50 | + |
| 51 | +Many internal details are hidden from this public API. For details on Kinect |
| 52 | +v2's USB protocols, depth decoding algorithms, calibration algorithms, and how |
| 53 | +to implement performance optimizers, you are encouraged to read the source |
| 54 | +code. The source code is the updated and authoritative reference for any |
| 55 | +functionalities. |
| 56 | + |
| 57 | +You can also see the following walkthrough for the most basic usage. |
| 58 | + |
| 59 | +Walkthrough |
| 60 | +=========== |
| 61 | + |
| 62 | +Here is an example to walk you through the API. See `examples/Protonect.cpp` |
| 63 | +for the full source. |
| 64 | + |
| 65 | +Headers |
| 66 | +------- |
| 67 | + |
| 68 | +First, include necessary headers. `registration.h` and `logger.h` are optional |
| 69 | +if you don't use them. |
| 70 | + |
| 71 | +@snippet Protonect.cpp headers |
| 72 | + |
| 73 | +Logging |
| 74 | +------- |
| 75 | + |
| 76 | +This shows how to set up the logger and logging level. |
| 77 | + |
| 78 | +@snippet Protonect.cpp logging |
| 79 | + |
| 80 | +Though @copydetails libfreenect2::createConsoleLoggerWithDefaultLevel |
| 81 | + |
| 82 | +You can implement a custom [Logger](@ref libfreenect2::Logger) and redirect |
| 83 | +%libfreenect2's log messages to desired places. |
| 84 | + |
| 85 | +Here is an example to save log messages to a file. |
| 86 | + |
| 87 | +@snippet Protonect.cpp logger |
| 88 | + |
| 89 | +And use it |
| 90 | + |
| 91 | +@snippet Protonect.cpp file logging |
| 92 | + |
| 93 | +%libfreenect2 uses a single global logger regardless of number of contexts and |
| 94 | +devices. You may have to implement thread safety measure in |
| 95 | +[log()](@ref libfreenect2::Logger::log), which is called from multiple threads. |
| 96 | +Console loggers are thread safe because `std::cout` and `std::cerr` are thread |
| 97 | +safe. |
| 98 | + |
| 99 | +Initialize and Discover Devices |
| 100 | +------------------------------- |
| 101 | + |
| 102 | +You need these structures for all operations. Here it uses only one device. |
| 103 | + |
| 104 | +@snippet Protonect.cpp context |
| 105 | + |
| 106 | +You must enumerate all Kinect v2 devices before doing anything else related to |
| 107 | +devices. |
| 108 | + |
| 109 | +@snippet Protonect.cpp discovery |
| 110 | + |
| 111 | +Also, you can create a specific [PacketPipeline](@ref libfreenect2::PacketPipeline) |
| 112 | +instead using the default one for opening the device. Alternatives include |
| 113 | +[OpenGLPacketPipeline](@ref libfreenect2::OpenGLPacketPipeline), |
| 114 | +[OpenCLPacketPipeline](@ref libfreenect2::OpenCLPacketPipeline), etc. |
| 115 | + |
| 116 | +@snippet Protonect.cpp pipeline |
| 117 | + |
| 118 | +Open and Configure the Device |
| 119 | +----------------------------- |
| 120 | + |
| 121 | +Now you can open the device by its serial number, and using the specific |
| 122 | +pipeline. |
| 123 | + |
| 124 | +@snippet Protonect.cpp open |
| 125 | + |
| 126 | +You can also open the device without providing a pipeline, then a default is |
| 127 | +used. There are a few alternative ways to [openDevice()](@ref libfreenect2::Freenect2::openDevice). |
| 128 | + |
| 129 | +After opening, you need to attach [Framelisteners](@ref libfreenect2::FrameListener) |
| 130 | +to the device to receive images frames. |
| 131 | + |
| 132 | +This [SyncMultiFrameListener](@ref libfreenect2::SyncMultiFrameListener) will |
| 133 | +wait until all specified types of frames are received once. Like loggers, you |
| 134 | +may also implement your own frame listeners using the same interface. |
| 135 | + |
| 136 | +@snippet Protonect.cpp listeners |
| 137 | + |
| 138 | +You cannot configure the device after starting it. |
| 139 | + |
| 140 | +Start the Device |
| 141 | +---------------- |
| 142 | + |
| 143 | +After finishing configuring the device, you can start the device. You must |
| 144 | +start the device before querying any information of the device. |
| 145 | + |
| 146 | +@snippet Protonect.cpp start |
| 147 | + |
| 148 | +You can [setIrCameraParams()](@ref libfreenect2::Freenect2Device::setIrCameraParams) |
| 149 | +after start if you have your own depth calibration parameters. |
| 150 | + |
| 151 | +Otherwise you can also use the factory preset parameters for |
| 152 | +[Registration](@ref libfreenect2::Registration). You can also provide your own |
| 153 | +depth calibration parameterss (though not color camera calibration parameters |
| 154 | +right now). Registration is optional. |
| 155 | + |
| 156 | +@snippet Protonect.cpp registration setup |
| 157 | + |
| 158 | +At this time, the processing has begun, and the data flows through the pipeline |
| 159 | +towards your frame listeners. |
| 160 | + |
| 161 | +Receive Image Frames |
| 162 | +-------------------- |
| 163 | + |
| 164 | +This example uses a loop to receive image frames. |
| 165 | + |
| 166 | +@snippet Protonect.cpp loop start |
| 167 | + |
| 168 | +[waitForNewFrame()](@ref libfreenect2::SyncMultiFrameListener::waitForNewFrame) |
| 169 | +here will block until required frames are all received, and then you can |
| 170 | +extract `Frame` according to the type. |
| 171 | + |
| 172 | +See libfreenect2::Frame for details about pixel format, dimensions, and |
| 173 | +metadata. |
| 174 | + |
| 175 | +You can do your own things using the frame data. You can feed it to OpenCV, |
| 176 | +PCL, etc. Here, you can perform registration: |
| 177 | + |
| 178 | +@snippet Protonect.cpp registration |
| 179 | + |
| 180 | +After you are done with this frame, you must release it. |
| 181 | + |
| 182 | +@snippet Protonect.cpp loop end |
| 183 | + |
| 184 | +Stop the Device |
| 185 | +--------------- |
| 186 | + |
| 187 | +If you are finished and no longer need to receive more frames, you can stop |
| 188 | +the device and exit. |
| 189 | + |
| 190 | +@snippet Protonect.cpp stop |
| 191 | + |
| 192 | +Pause the Device |
| 193 | +---------------- |
| 194 | + |
| 195 | +You can also temporarily pause the device with |
| 196 | +[stop()](@ref libfreenect2::Freenect2Device::stop) and |
| 197 | +[start()](@ref libfreenect2::Freenect2Device::start). |
| 198 | + |
| 199 | +@snippet Protonect.cpp pause |
| 200 | + |
| 201 | +Doing this during `waitForNewFrame()` should be thread safe, and tests also |
| 202 | +show well. But a guarantee of thread safety has not been checked yet. |
| 203 | + |
| 204 | +THE END. |
| 205 | +*/ |
0 commit comments