While companies like Amazon and Quiet Logistics already employ robots to operate their warehouses, most of the wholesale distributors still rely on manual labor and only start looking more closely at logistics automation solutions. The idea to make machines do at least some of warehouse work is tempting, because managing a huge storage is a demanding task, and human workers often fall short of that. Doing dull and monotonous work, such as counting boxes on a pallet, people easily get tired and start making mistakes. Besides that, human operators, unfortunately, are not completely reliable. This results in losses both in money and time, which is something everyone prefers to avoid.
So, what if we decide to introduce automation into this process and count cartons using a camera and image analysis software? When appropriately programmed, machines work quickly and efficiently. They neither get tired nor try cheating. Everything they need to do their job is an instruction, either programmed directly or derived from a machine learning process. And if we want this instruction to be accurate, the images to analyze should be of reasonable quality and contain all the information required.
What You Will Learn
Possible machine vision solutions
The basic image analysis strategy, in the case of counting boxes, will include several iterations of image segmentation: first, to separate a pallet with boxes stacked on it from the surrounding scene (which may in its turn include other packages irrelevant to this particular pallet), then to separate each box from all others on the pallet and from the pallet itself. To perform this task successfully, the software should find all the edges of cartons correctly and do not mistake for box edges any other features found.
Training a neural network
Today, several quite accurate edge detecting techniques exist, such as Canny edge detector, Sobel operator, Marr-Hildreth algorithm, as well as their modifications and combinations. These and other algorithms can be used as layers of a convolution neural network (CNN). Deep neural networks represent a set of state-of-the-art techniques used for data analysis and classification, and CNNs show particularly great results in image analysis. Neural networks are trained with huge sets of images similar to those they will eventually interpret. This information allows them to build a pattern of features corresponding to the objects they are learning to recognize.
Training a CNN to calculate boxes on the pallet will require thousands of images to make its predictions as accurate as a human?s, and the training set can be continuously extended to increase the output accuracy.
Calculating boxes based on the volume
Another possible approach requires adding one more axis to the image. It assumes that the boxes on the pallet are of the same type and occupy the same amount of space. It also involves 3D reconstruction of the scene and accordingly requires a stereo camera and infrared proximity sensors.
The idea is based on calculating the volume occupied by the entire loaded pallet, then the boxes only (pallet and in-between volume extracted), and finally the volume of a single box. Knowing the volume occupied both by all the boxes and one box, the machine can easily find the number of boxes on a pallet:
Machines will suffer
In the perfect warehouse world where all pallets are stacked according to existing TI-HI standards, image analysis algorithms show their best. Unfortunately, it is not always the case. In real life, several challenges emerge that can by far influence the performance of the image analysis software:
- Inaccurate palletizing: if there is no order in a way boxes are stacked, some of them may be impossible to detect even when through both top and lateral views.
- Illumination conditions: the insufficient amount of light produces noise and makes edge detection more difficult.
- Shadows: they can be misinterpreted as box edges.
- Wrapping film: it reflects light and obscures box shapes.
- Different types of packaging: variations in shape, size, and color of packages should be minimal for software to recognize and interpret them as objects of the same type.
Accuracy is everything
Making machine vision a part of the warehouse management system can bring a significant return on investment in automation. But to maximize the profit, a logistics company should revise the existing palletizing operations in a way that would improve the efficiency of the automated visual inspection software.
The more standardized warehouse operations become, the more of them can be performed by machines. In the future, we can expect the advent of fully automated warehouses, where all the loading, sorting and counting is performed by robots.
Automated visual inspection
Dissatisfied with the speed, results or costs of your inspection process? Consider what changes automated visual inspection can bring.
Show me more info