A First Blueprint for Machine Vision: Look, Record, then Perfect
An Introduction to Machine Vision
Machine vision (also called “industrial vision” or “vision systems”) is the use of digital sensors (wrapped in cameras with specialized optics) that are connected to processing hardware and software algorithms to visually inspect…pretty much anything. Machine vision is a true multi-disciplinary field, encompassing computer science, optics, mechanical engineering, and industrial automation. While historically the tools of machine vision were focused on manufacturing, that’s quickly changing, spreading into medical applications, research, and even movie making.
A machine vision system combines image capture and processing systems (computer hardware and software) with digital input/output devices (cameras) and computer networks (to store and share the image data) to drive real-time quality control systems, or to direct equipment, such as manufacturing robots. This is slightly different from “computer vision,” which is primarily about image processing. Instead, machine vision systems are built for visual inspection and control under demanding industrial applications that require high-speed, high-magnification, 24-hour operation, and/or repeatable measurements.
Doing a Thankless Job Really Well
No machine beats human vision for versatility, but other human weaknesses limit their productivity in a manufacturing environment. Because these kinds of jobs are “no fun.” Boredom, distraction, and fatigue all typically degrade human performance in vision-related factory tasks like inspection, which require high speeds, constant operation and a lot of monotony.
Factory automation utilizing a machine vision system in such tasks, then, offers many benefits. Machine vision systems can perform repetitive tasks faster and more accurately, with greater consistency over time than humans. They can reduce labor costs, increase production yields, and eliminate costly errors associated with incomplete or incorrect assembly. They can help automatically identify and correct manufacturing problems on-line by forming part of the factory control network.
This can be a part of the inspection process itself (e.g. checking a measurement or identifying whether a character string is printed correctly) or through some other input needed for control (e.g. robot control or type verification). The machine vision system can include almost any number of cameras, all capturing, interpreting and signaling individually to a central control system according to a pre-determined tolerance or requirement.
This makes it easy to automate complex multi-stage visual inspection tasks, or simply perform simple inspections at speed and scale. These kinds of applications include positioning, identification, verification, measurement, and flaw detection. Machine vision system has driven ever higher standards in manufacturing— improved product quality, higher yields and lower production costs. The net result is greater productivity and improved customer satisfaction through the consistent delivery of quality products.
Where Do You Want to Go Next?
Because machine vision systems are used literally everywhere—in factories, on the road, and in space—the pieces they are built from need to be standardized but customizable. In general, each component of a machine vision system does one thing. This makes it easier to swap parts to add a new capability or level of performance without replacing parts unnecessarily. It also makes it simple to replace a part that has failed. Cameras have gone from the weakest link in the chain to the pixel-pushing drivers of innovation in the rest of the system. This is in contrast to a consumer-level digital camera or even digital film camera, which will record light, but also perform different degrees of processing to improve image quality or reduce file sizes as well as record out to a transferrable medium.
Elements of Your Awesome Vision System
A typical machine vision system will be part of an automated production process consisting of the following components:
Camera: Systems will have one or more digital cameras that record color, monochrome, or wide-spectrum images. Just like consumer cameras, the optics/lenses attached deliver a specific field of view and available light. To pick the proper lens you will first need to know the field-of-view (FOV) and the working distance. The FOV is the size of the area you want to capture. The working distance is approximately the distance from the front of the camera to the part being inspected. A more exact definition takes into account the structure of the lens.
Trigger: This component makes sure that specific action takes place in response to something the camera sees. A synchronizing sensor for part detection (often an optical or magnetic sensor) to trigger image acquisition and processing and some form of actuators to sort, route or reject defective parts
Image Processor: A computer program (normally running on a version of Microsoft Windows) to process images, detect, measure, compare etc. in order to confirm a quality criteria has been met or to provide type verification or robot control to another control system.
Cables: Input/output hardware (e.g. digital I/O) or communication links (e.g. Ethernet, USB, Firewire, CameraLink, etc.) to report results and to automatically reject components.
Building Your Own System
Any machine vision system consists of a few critical components. The image sensor/ camera captures light and converts it into an electrical signal that computers can understand and sends it to a processing engine that renders and communicates the result to a computer that can do something with it.
While the additional features and operational requirements depend on the application, all machine vision systems share some fundamental attributes and behaviors. Systems all have a need to image or inspect a scene or object, operating on a continuous basis at the fastest practical speed.
The essential elements of an inspection system, shown in Figure 2, include a delivery vehicle, the vision system, the response system, and sensors to trigger image capture and system response. The delivery vehicle positions the object for inspection. The vision system, which includes camera, optics, lighting, and image processor, captures and processes the object image to determine a pass/fail response. The response system takes the required action as well as communicating results to operators or other systems. The sensor triggers a response, identifying when the object is positioned properly for the system to perform their tasks.
The first, and arguably most important, step in developing an inspection system is positioning or “staging.” After all, no amount of processing can retrieve information that a sensor didn’t capture at the scene. This is the determination of how a subject will be situated in front of the camera for imaging. Figuring out how objects will be delivered to the camera and sensor can also be the hardest part of machine vision design. The delivery choice will frame all the remaining system choices, including camera, lighting, sensors, and response systems.
And it seems like the camera can go almost anywhere. Let’s look at a few extreme examples. Instead of objects being delivered via conveyer belt in a tidy and predictable fashion, they could run or even fall in.
In the case of systems built by Replay Technologies for their stadium systems, the target is in the middle. It can be used to capture an entire football or basketball game. or it could be a James Bond skydiving scene in Quantum of Solace. In the latter, the cameras were in a ring around a wind tunnel to ensure that action was captured from multiple angles.
The system in Figure 4, works for filming movie stunts, reversing the motion relationship that typical vision assumes: here the camera the thing that is moving. That which is being filmed, namely the motorcycle and rider, does not move in relationship to the camera. The other subject being recorded, the scenery is fixed. It’s the camera that is moving at 240km/hour.
Getting Better All the Time
It’s not just what’s possible—it’s what you do with it.
Strong industry innovation and competition have driven dramatic improvements in what cameras, processing, and automation can do. Visual inspection makes it possible to precisely guide robots and other machinery to create more adaptable manufacturing processes. Custom products and more variable production environments are possible when machines can see as well as build and much of this is taking place on line, making for a more efficient and faster inspection and eliminating waste or scrap material as they run.
With a never-ending thirst for improved quality and the availability of higher resolution cameras with faster and more efficient processing power, increasing bandwidth, and access to lower cost cabling and storage, there truly is no limit to where one might find what started as machine vision. The sky (and the universe) truly is just the beginning.
Five CMOS Camera Developments to Watch in 2014 
When Can a Semi-Custom Vision System Help? 