EasyVision & ComfyCV: A Modular AI Toolkit for Computational Perception in the Arts
Golan Levin, Claire Vlases, Lorie Chen • June 2026
This project was developed during the CMU College of Fine Art's 2026 AI Toolmaking Residency to support Experimental Capture, an interdisciplinary studio course in which students build systems for sensing and representing phenomena beyond the limits of ordinary human perception. Rather than treating AI primarily as a tool for image generation, the project reframes it as a framework for perception, measurement, and analysis. A central assignment in the course, Typology Machine, asks students to construct a system that automatically collects, regularizes, and compares observations in order to investigate a research question. The residency sought to lower the technical barriers to this kind of inquiry by providing students with open, extensible computational instruments that allow them to build, modify, and compose their own workflows from state-of-the-art computer vision research, rather than simply consume fixed applications.
The project adopts ComfyUI as its primary development environment because it occupies a unique position within the creative AI ecosystem. Its node-based programming interface is already familiar to many artists through environments such as TouchDesigner, Max/MSP, and Grasshopper, giving it a low floor for newcomers while remaining flexible enough to support sophisticated research workflows. At the same time, the ComfyUI community rapidly wraps newly published computer vision research into reusable nodes, dramatically shortening the path from research laboratories to the classroom. Compared to commercial vision platforms such as Roboflow, ComfyUI offers a low-cost, open-source, and highly customizable alternative in which educators and students retain control over the entire computational pipeline and can inspect, modify, and extend every stage of the process.
Rather than producing a single application, the residency resulted in an ecosystem of interoperable tools. The EasyVision suite includes browser-based utilities such as EasyLabeler for annotating images and video; EasyTrain for training custom YOLO detectors from those annotations;
EasyDetect for detecting phenomena using custom-trained YOLO models together with powerful foundation models; and EasyTrack, for compiling and tracking detections, and exporting the results for downstream creative tools. Alongside these are ComfyCV, a curated collection of documented ComfyUI workflows for contemporary computer vision tasks including open-vocabulary detection, segmentation, contour extraction, pose estimation, monocular depth estimation, semantic saliency, concept activation heatmaps, image similarity analysis, and other forms of computational perception. The project also includes original custom ComfyUI nodes that introduce a unified TRACKS data abstraction, allowing outputs from detectors and trackers such as SAM3, LocateAnything, YOLO, and CoTracker to interoperate through a common representation and be exported for downstream visualization, animation, analysis, and creative coding.
A substantial portion of the residency focused not on inventing new AI models, but on making existing research systems installable, reproducible, interoperable, and teachable for artists. This required building original software, designing coherent educational workflows, solving practical compatibility problems across rapidly evolving research code, writing installation patches where necessary, and producing extensive classroom-oriented documentation. The result is a collection of pedagogical "software instruments" that expose sophisticated computer vision capabilities through transparent, modular workflows while remaining accessible to students with varied technical backgrounds. Equally important, the toolkit emphasizes conceptual understanding by clearly distinguishing detection, segmentation, tracking, identity, and representation, encouraging students to think critically about what each computational process is actually measuring.
The long-term goal extends beyond the software itself. EasyVision and ComfyCV are intended to shorten the distance between computer vision research and arts education by transforming emerging research models into approachable, documented, interoperable educational resources. Within Experimental Capture, these tools become the foundation for students' own "typology machines": computational systems that collect, regularize, analyze, and compare phenomena in order to formulate and investigate research questions through systematic observation rather than image generation. By making computational perception accessible to an entire classroom without requiring scarce specialized hardware or extensive software engineering, the project expands students' ability to use AI not simply to synthesize media, but to develop new ways of seeing, measuring, and understanding the world.
- Reframed AI as a medium for computational perception rather than image generation within arts education. Developed an open, modular ecosystem of annotation, training, detection, segmentation, tracking, and visualization tools centered on ComfyUI.
- Introduced a common TRACKS representation that unifies outputs from multiple contemporary computer vision systems.
- Lowered the barrier between cutting-edge computer vision research and studio practice through documentation, workflow design, software integration, and open-source release.
- Created a practical foundation for students to build computational "typology machines" that use AI for observation, analysis, and creative inquiry.