Welcome to the first 🏆 and only 🥇 repo integrating FLIR's 🔥 Spinnaker SDK 💕 with YOLO for live inference 🚀

FLIR cameras are the go-to choice for high-performance imaging. They offer a wide range of features that span from high speed, high resolution, and thermal imaging. YOLO is the leading computer vision algorithm. It is fast, accurate, and well supported. When these two technologies are combined they create a powerful and versatile tool for almost any CV application.

YOLO supports RTSP and USB webcam streams straight from the GitHub repo. This severely limits the quality of images/video that can be used for live inference, and my desire to control the imaging source to increase performance led me down a rabbit hole to find a readymade solution that utilized FLIR’s Spinnaker SDK in the YOLOv8 architecture.

I am proud to be able to share with the community my GitHub fork of the YOLOv8 repo featuring integration with FLIR cameras through the Spinnaker SDK.

Documentation

Documentation

See below for installation and usage for Spinnaker YOLO example, and see the YOLOv8 Docs for full documentation on training, validation, prediction and deployment.

Install

Environment requirements outside of Ultralytics YOLO:

1. Python==3.8.10
2. CUDA Toolkit==11.8
3. PyTorch>=2.0
4. Spinnaker SDK == Python38

#Now install Ultralytics
pip install ultralytics

#Change directory to downloads folder
cd ~/Downloads

#Clone this repo that contains the FLIR compatible files
git clone https://github.com/laminarize/FLIR-Spinnaker-YOLO.git

Change directory in terminal to your Python Site Packages folder which will contain the Ultralytics folder. - Depending on operating system, but generally "...Python\Python38\Lib\site-packages\ultralytics"

#Once you have changed your working directory to your Python Site Packages path folder - Delete Ultralytics folder that was installed in the Site-Packages folder from the "pip install ultralytics" command, then copy GitHub cloned Ultralytics folder
rm -r ./Ultralytics
cp -r ~/Downloads/FLIR-Spinnaker-YOLO/Ultralytics ./

Usage

CLI

YOLOv8 may be used directly in the Command Line Interface (CLI) with a yolo command. Source 0 has been redirected to stream FLIR camers using the Spinnaker SDK. The vid_stride argument has been converted to use the metadata of the instantiated Spinnaker camera object instead of openCV:

#Change directory to the yolo folder within the Ultralytics site-packag e then you can run the modified yolo code like so:
yolo predict model=yolov8n.pt source=0 show=True save=False vid_stride=4 spectrum='visible'

Remaining GitHub content directly from Ultralytics GitHub:

yolo can be used for a variety of tasks and modes and accepts additional arguments, i.e. imgsz=640. See the YOLOv8 CLI Docs for examples.

Python

YOLOv8 may also be used directly in a Python environment, and accepts the same arguments as in the CLI example above:

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n.yaml")  # build a new model from scratch
model = YOLO("yolov8n.pt")  # load a pretrained model (recommended for training)

# Use the model
model.train(data="coco128.yaml", epochs=3)  # train the model
metrics = model.val()  # evaluate model performance on the validation set
results = model("https://ultralytics.com/images/bus.jpg")  # predict on an image
path = model.export(format="onnx")  # export the model to ONNX format

Models download automatically from the latest Ultralytics release. See YOLOv8 Python Docs for more examples.

Models

YOLOv8 Detect, Segment and Pose models pretrained on the COCO dataset are available here, as well as YOLOv8 Classify models pretrained on the ImageNet dataset. Track mode is available for all Detect, Segment and Pose models.

All Models download automatically from the latest Ultralytics release on first use.

Detection

See Detection Docs for usage examples with these models.

Model	size (pixels)	mAPval 50-95	Speed CPU ONNX (ms)	Speed A100 TensorRT (ms)	params (M)	FLOPs (B)
YOLOv8n	640	37.3	80.4	0.99	3.2	8.7
YOLOv8s	640	44.9	128.4	1.20	11.2	28.6
YOLOv8m	640	50.2	234.7	1.83	25.9	78.9
YOLOv8l	640	52.9	375.2	2.39	43.7	165.2
YOLOv8x	640	53.9	479.1	3.53	68.2	257.8

• mAP^val values are for single-model single-scale on COCO val2017 dataset.
  Reproduce by yolo val detect data=coco.yaml device=0
• Speed averaged over COCO val images using an Amazon EC2 P4d instance.
  Reproduce by yolo val detect data=coco128.yaml batch=1 device=0|cpu

Segmentation

See Segmentation Docs for usage examples with these models.

Model	size (pixels)	mAPbox 50-95	mAPmask 50-95	Speed CPU ONNX (ms)	Speed A100 TensorRT (ms)	params (M)	FLOPs (B)
YOLOv8n-seg	640	36.7	30.5	96.1	1.21	3.4	12.6
YOLOv8s-seg	640	44.6	36.8	155.7	1.47	11.8	42.6
YOLOv8m-seg	640	49.9	40.8	317.0	2.18	27.3	110.2
YOLOv8l-seg	640	52.3	42.6	572.4	2.79	46.0	220.5
YOLOv8x-seg	640	53.4	43.4	712.1	4.02	71.8	344.1

• mAP^val values are for single-model single-scale on COCO val2017 dataset.
  Reproduce by yolo val segment data=coco.yaml device=0
• Speed averaged over COCO val images using an Amazon EC2 P4d instance.
  Reproduce by yolo val segment data=coco128-seg.yaml batch=1 device=0|cpu

Classification

See Classification Docs for usage examples with these models.

Model	size (pixels)	acc top1	acc top5	Speed CPU ONNX (ms)	Speed A100 TensorRT (ms)	params (M)	FLOPs (B) at 640
YOLOv8n-cls	224	66.6	87.0	12.9	0.31	2.7	4.3
YOLOv8s-cls	224	72.3	91.1	23.4	0.35	6.4	13.5
YOLOv8m-cls	224	76.4	93.2	85.4	0.62	17.0	42.7
YOLOv8l-cls	224	78.0	94.1	163.0	0.87	37.5	99.7
YOLOv8x-cls	224	78.4	94.3	232.0	1.01	57.4	154.8

• acc values are model accuracies on the ImageNet dataset validation set.
  Reproduce by yolo val classify data=path/to/ImageNet device=0
• Speed averaged over ImageNet val images using an Amazon EC2 P4d instance.
  Reproduce by yolo val classify data=path/to/ImageNet batch=1 device=0|cpu

Pose

See Pose Docs for usage examples with these models.

Model	size (pixels)	mAPpose 50-95	mAPpose 50	Speed CPU ONNX (ms)	Speed A100 TensorRT (ms)	params (M)	FLOPs (B)
YOLOv8n-pose	640	50.4	80.1	131.8	1.18	3.3	9.2
YOLOv8s-pose	640	60.0	86.2	233.2	1.42	11.6	30.2
YOLOv8m-pose	640	65.0	88.8	456.3	2.00	26.4	81.0
YOLOv8l-pose	640	67.6	90.0	784.5	2.59	44.4	168.6
YOLOv8x-pose	640	69.2	90.2	1607.1	3.73	69.4	263.2
YOLOv8x-pose-p6	1280	71.6	91.2	4088.7	10.04	99.1	1066.4

• mAP^val values are for single-model single-scale on COCO Keypoints val2017 dataset.
  Reproduce by yolo val pose data=coco-pose.yaml device=0
• Speed averaged over COCO val images using an Amazon EC2 P4d instance.
  Reproduce by yolo val pose data=coco8-pose.yaml batch=1 device=0|cpu