[Home][Technologie][Werkwijze]

Author: Holger Kattentidt, Marshalllaan 2, 2625GZ Delft, The Netherlands

General description:

The focus of solving an inspection task is not longer software development. The newly developed software Sort4all reduces risks of new applications in the area of high speed inspection tremendously. A smart approach combines interpreted commands with a fully functional framework. This approach moves the focus from the problems of software development towards solving the inspection tasks. It is much easier to test and apply new sensor applications. New sensor types, inspection tasks, and future solutions can be applied easily.

High speed inspection systems

In a nutshell, a high speed inspection system controls sensors, reads out sensor data, carries out raw data processing, and analyzes measured and calculated data streams. Objects have to be found, regions of interest, or significant segments in images. These areas are classified in order to control other processes (e.g. actuators for sorting)

Applications

Typical applications are sensor controlled sorting, sensor controlled quality control, and sensor controlled process control. Applications can be found in in processing of bulk streams or processing of positioned objects, inspecting products as minerals, stones, gravel, ores, coal, nuts, sweets, vegetables, pies, beans, carrots, potatoes, potato chips, broccoli, celery, all kinds of fruit, tobacco, paper, textile, metals, plastics, glass, paper, demolition waste, and so on. Inspection systems look for e.g. defects, contaminations, discolored areas or misshaped objects. Sorting may be done e.g. by an array of air jets, which blows special objects out of a falling stream of material (see figure HSI_System.jpg).

The basic system

The hart of an inspection system is the software. Sort4all provides the user with general multi purpose high speed inspection software. The software contains all main tasks of such a system, as: timing, synchronization, communication, user access rights, graphical user interface for touch screens, computer networking. Further it contains extendable sets of filters and algorithms for raw data processing, image segmentation, object analysis, and object classification. This framework is called the ‘kernel’. The basic system is provided as one compiled executable per computer. Although working on a network of computers, the user has the feeling to work on one system which is directly connected to all sensors in the system. The basic system is fast enough to analyze and sort up to 900 objects per second and to react within 5 milliseconds on particles moving with up to 20 meter per second.

Specializing the system to a solution

With Sort4all, the user first connects the sensors into the framework and specifies the sorting task by calling filters, calculations and classifications on the data streams and segmented areas. He also defines what to do with the result of the classification. Then these definitions and commands are executed by embedded interpreters in the kernel. In this way, the basic system is adjusted to a special inspection task just by adding function calls, commands and if needed calculations in an interpreter language (Tcl). Supported by a library of interpreter commands for controlling and specializing, all parts of a special inspection task can be designed and changed in very short time. The user can start very fast the inevitable testing and improvement cycles to match the task of his customer.

From bits to objects

The high level commands which are handled in the interpreter help to think and program in ‘objects’ or ‘contaminated areas’ instead of concentrating on processing of bits and bytes. The main advantage is that the user can focus directly on the task he wants to solve, instead of programming. Data streams, bits and bytes are handled within the kernel. They are not visible for the user. However, most of them are accessible through the interpreters, if needed. Applications turn out to be reduced to interpreted configuration scripts. In that way, a break through in programming of sensor based applications is achieved

Embedded interpreter

The component processor/computer speed may be critical in a high speed inspection system. Therefore all time consuming calculations are executed in the compiled kernel. The embedded interpreter only controls the setup of the system and the classification process. On one hand, the interpreter hides and accelerates complex calculations. On the other hand, the free programmable embedded interpreter enables the user to run the most complex calculations or classifications also on the interpreter level if needed.

Setup of the system

Adapting the basic system to a special task is straight forward process: first, declare the hardware, computers, sensors, and devices in your system. Then, declare users, access rights, and which pages of the graphical user interface for system control you want to use. Now, declare the filters on the measured data, color classes, and the main parameters used by the system. Then, you tell the system in a few interpreter lines, how to segment objects or areas, how to calculate features of these areas, and how to classify them in e.g. good or bad (see figure SampleScript.jpg). Further, you may give some instructions about doing actions as blowing on bad particles, declare and define statistics, activity histograms, alarm definitions and reactions on alarms, system calibration, sensor controls, the active display language (Unicode) and much more (see figure HighSpeedInspectionMadeEasy.jpg.

Synchronization

Just like that, it is not possible to synchronize sensor data after delayed communication with unknown and changing delay times. The basic system utilizes a patented method (WO2005083367), which allows synchronization between asynchronous sensor data streams and actuators with a precision of at least the line scan rate of the sensors in the system. This even works in an asynchronous coupled network of computers (e.g. standard Ethernet). Even data from different (free running) sensors can be combined and synchronized by this patented synchronization method.

Interfaces and configurations

As far as available, the basic system uses industrial standards (computers, interfaces, protocols, operating system) in order to be compatible to future releases of the peripheral systems. This ensures reliability and support on all used components. Interfaces of drivers for frame grabbers, digital and analog i/o interfaces, serial and parallel ports are integrated in the kernel. Interfaces, which are not yet implemented, will be implemented on demand. Therefore, almost every sensor with an XP-driver can be connected to the kernel. Usually, for every product type a special configuration is needed. The user can switch between different individual configurations and classification tasks within seconds. A large example library of applications provides methods for e.g. automatic adaptation on changing products, neural network classifiers, or Bayes classifiers.

Offline development

Based on Windows XP embedded, the basic system is suited to run in embedded systems and on Windows XP/Vista computers as well. The basic system can run e.g. on a desktop computer without using sensors and other communication hardware. Sort4all makes it possible to grab and file sensor data or regions of interest in measured data streams. After reloading the data offline, they can be processed like online measured data. Filters and classifications can be developed, improved and tested on a desktop computer on real data. Statistical analysis may be applied to find the best solution. The desktop may also be remotely connected to the computers in the machine. In this way, remote updates, remote improvements, remote service, and remote control of the system quality become standard for high speed inspection systems.

Complex solutions

The system can be used for fast development of simple solutions, for development of flexible solutions and for development of complex solutions as well. With more efforts usually a better classification is achieved. This makes it very easy to test new sensor types, new sorting algorithms, new lighting, new product streams, and new inspection solutions.

Management tool

The management tool enables the management of the factory to read out e.g. the history of the process data, statistics and eventually alarms of the applications in a factory. This information can be filed or processed in a further step.

State of the art and advantages of this new technology

High speed inspection often utilizes line scan cameras on PC-based systems. The processing time of algorithms in high speed inspection systems is directly related to the available computing power. Therefore a fast implementation and adequate manipulation of raw image data, color classes and classification is a key issue in high speed inspection. Although the basic process of high speed inspection is the same in almost all applications, Sort4all is the first general purpose software available on the market.

Framework instead of a toolbox

In the state of the art, machine vision is based on “image delivering systems”. In most cases you get some sort of a vision tool box with tools for image capturing, image processing, image analysis, and image or area classification. The user writes software which combines these tools to a continuously working application. Usually knowledge and experience in C++, multiprocessing, operating systems, PCI-bus control, and communication protocols is needed. Software development is not the core business of a machine builder. Many system developers will agree that developing software with toolboxes takes at least twice the time and twice the resources as expected. Sort4all, however, is much more than a toolbox or an image delivering system. It is a fully functional framework, running in a computer network. This framework is ready to solve automated vision tasks with line scan sensors of any kind. Because the system is fully functional, the developer can focus directly on the task he wants to solve. This may be compared with the step from programming a table calculation in C++ towards doing it with Microsoft Office Excel.

Avoiding unpredictable risks

Nowadays sensor systems may stick in their specialization. Focused and optimized on a special group of tasks they are not designed for easy reuse in new applications. Any change may lead to reprogramming and recompilation, often accompanied by unpredictable risks and development time. Sort4all, however, is designed to be a general purpose platform which is specialized by the developer’s scripts. While the platforms guarantees downward compatibility for many years, changes in script will not need a change in the platform. Sometimes, a later version of the platform is needed when newly developed sensors or interfaces have to be utilized. Therefore, updates of the basic system will be available for downloading.

Shorter time to market

There is no standard software available on the market for fast and easy testing of new sensors, new sensor combinations or new classification ideas. In first instances, the quality of a sensor controlled system strongly depends on the quality of the sensors. However, only working sensor applications deliver an added value which can pay back the investment. Sort4all is a synonym for fast application development. This results in a shorter time to market for all involved parts of the system.

Main advantage

The advantage of the innovation Sort4all is that it helps to solve inspection tasks on a much easier level. Rapid prototyping, fast tests, improvements, and more diversity become possible with Sort4all. Skills in a higher programming language or compilation of source code are not longer needed. No special hardware is needed to reach the necessary calculation power, because Sort4all utilizes a network of computers, linked by Ethernet, to gain scalable image processing power. Offline development helps to find solve problems and to improve classification.

Innovation in image processing

The expected relevance and benefit of the innovation for machine vision is very high. The potential of the innovation to increase the number of high speed inspection applications and to widen the spectrum of sensor applications is obvious. More complex solutions become possible. This results in new markets, further innovations and improvements in inspection quality. Another benefit is the shift of the developer’s perception from solving software problems towards creating solutions for his customers. Last, but not least, less risks and higher quality of solutions may be mentioned as main benefit for the branch.

Quality improvement

In a straight forward process the basic system is specialized to a certain task. Fast prototyping and fast testing will result in a shorter time to market. More advanced applications and solving new inspection tasks become possible. The basic system is tested with different CCD line scan cameras, x-ray sensors and arrays of metal detectors. Other sensors like microwave sensors, near and far infrared sensors, ‘smell’ measurement, acoustic measurement, ultraviolet sensitive or fluorescence measurement may also be adapted. The range of applicable sensors is almost unlimited. In principle, any line scan sensor can be adapted to the kernel.

Shifting perception

Because the process of high speed inspection is very general, a framework could be programmed as a basis for any task in high speed inspection with line scan sensors. It hides all basic components as communication, data grabbing, data flow, data input / output, timing, synchronization, multi tasking, network communication etc. from the developer. This moves the focus away from programming towards solving the customer’s tasks.

More reliability and less risks

The basic system is tested in many applications, e.g. sorting of tobacco, carrots, beans, pies, celery slices, sweets, recycling glass. This increases the reliability, component capability and usability in new sorting or measurement tasks.

Standards

Improving sensor controlled sorting, quality control, andprocess control is a main issue in machine vision. Sort4all is a major component in sensor controlled systems. This software component becomes increasingly complex in nowadays applications, especially when running on a network of computers. There is only one way to handle the risks of software development: create standards, and use them. Only standards can form a reliable basis for further development. Where possible, Sort4all uses standards which results in reliability and less risks using the platform.

 

 

[Up][Next]