From Wikipedia, the free encyclopedia
Emerging class of microprocessor
A vision processing unit (VPU ) is (as of 2023) an emerging class of
microprocessor ; it is a specific type of
AI accelerator , designed to
accelerate
machine vision tasks.
[1]
[2]
Overview
Vision processing units are distinct from
graphics processing units (which are specialised for
video encoding and decoding ) in their suitability for running
machine vision algorithms such as CNN (
convolutional neural networks ), SIFT (
scale-invariant feature transform ) and similar.
They may include
direct interfaces to take data from
cameras (bypassing any off chip buffers), and have a greater emphasis on on-chip
dataflow between many
parallel execution units with
scratchpad memory , like a
manycore
DSP . But, like video processing units, they may have a focus on
low precision
fixed point arithmetic for
image processing .
Contrast with GPUs
They are distinct from
GPUs , which contain specialised hardware for
rasterization and
texture mapping (for
3D graphics ), and whose
memory architecture is optimised for manipulating
bitmap images in
off-chip memory (reading
textures , and modifying
frame buffers , with
random access patterns ). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.
Target markets are
robotics , the
internet of things (IoT), new classes of
digital cameras for
virtual reality and
augmented reality ,
smart cameras , and integrating machine vision acceleration into
smartphones and other
mobile devices .
Examples
Movidius Myriad X , which is the third-generation vision processing unit in the Myriad VPU line from
Intel Corporation .
[3]
Movidius Myriad 2 , which finds use in
Google Project Tango ,
[4]
Google Clips and DJI drones
[5]
Pixel Visual Core (PVC), which is a fully programmable
Image , Vision and
AI processor for mobile devices
Microsoft HoloLens , which includes an accelerator referred to as a holographic processing unit (complementary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking and vision for augmented reality applications.
[6]
Eyeriss , a design from
MIT intended for running
convolutional neural networks .
[7]
NeuFlow , a design by
Yann LeCun (implemented in
FPGA ) for accelerating
convolutions , using a dataflow architecture.
Mobileye EyeQ , by
Mobileye
Programmable Vision Accelerator (PVA), a
7-way VLIW Vision Processor designed by
Nvidia .
Broader category
Some processors are not described as VPUs, but are equally applicable to machine vision tasks. These may form a broader category of
AI accelerators (to which VPUs may also belong), however as of 2016 there is no consensus on the name:
See also
Adapteva Epiphany , a manycore processor with similar emphasis on on-chip dataflow, focussed on 32-bit floating point performance
CELL , a multicore processor with features fairly consistent with vision processing units (SIMD instructions & datatypes suitable for video, and on-chip DMA between scratchpad memories)
Coprocessor
Graphics processing unit , also commonly used to run vision algorithms. NVidia's Pascal architecture includes FP16 support, to provide a better precision/cost tradeoff for AI workloads
MPSoC
OpenCL
OpenVX
Physics processing unit , a past attempt to complement the CPU and GPU with a high throughput accelerator
Tensor Processing Unit , a chip used internally by Google for accelerating AI calculations
References
^ Seth Colaner; Matthew Humrick (January 3, 2016).
"A third type of processor for AR/VR: Movidius' Myriad 2 VPU" . Tom's Hardware .
^ Prasid Banerje (March 28, 2016).
"The rise of VPUs: Giving Eyes to Machines" . Digit.in .
^
"Intel® Movidius™ Vision Processing Units (VPUs)" . Intel .
^ Weckler, Adrian.
"Dublin tech firm Movidius to power Google's new virtual reality headset" . Independent.ie . Retrieved 15 March 2016 .
^
"DJI Brings Two New Flagship Drones to Lineup Featuring Myriad 2 VPUs - Machine Vision Technology - Movidius" . www.movidius.com .
^ Fred O'Connor (May 1, 2015).
"Microsoft dives deeper into HoloLens details: 'Holographic processor' role revealed" . PCWorld .
^ Chen, Yu-Hsin; Krishna, Tushar; Emer, Joel &
Sze, Vivienne (2016).
"Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks" . IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers . pp. 262–263.
^
"Introducing Qualcomm Zeroth Processors: Brain-Inspired Computing" . Qualcomm . October 10, 2013.
^
"Intel to Bring a 'VPU' Processor Unit to 14th Gen Meteor Lake Chips" . PCMAG .
External links
Differentiable computing
General Concepts Applications Hardware Software libraries Implementations
Audio–visual Verbal Decisional
People Organizations Architectures
Portals
Categories