Real know-how when picking the unknown

Robust and flexible robotic automation solutions are becoming a ‘must have’ right across logistics

The global pandemic has radically impacted the supply chain and logistics industry, making the need for robotic automation more urgent than ever

With more than 70% of labour in warehousing being dedicated to picking and packing, numerous companies are gradually investing in logistics automation. But what happens when the robots must handle an unlimited number of (unknown) stock-keeping units? These companies need a fast, reliable, and robust way to automate picking and placing of a large variety of objects. 

Automated vision solution

This challenge was taken up successfully by Dutch company Fizyr. The computer vision company, based in Delft, focuses on enabling robots to pick unknown objects, even in harsh logistics environments. The result is an automated vision solution that enables logistic automation in various conditions and applications, such as item picking, parcel handling, depalletising, truck unloading or baggage handling. To complete the system with the optimal hardware, Fizyr integrates compact, robust Ensenso 3D cameras, in combination with high performance GigE uEye cameras from IDS.

Instead of providing a proprietary picking cell, as many players in the market do, Fizyr has created a plug-and-play modular software product that integrates smoothly with any system, giving integrators the freedom to choose the best hardware (eg, Cobots or Industrial robots) for their picking cell. While facilitating cost, robustness, and speed optimisation, IDS is the preferred partner, in terms of cameras. In essence, the camera sees products to be picked or classified. Depending on the individual customer application, up to four Ensenso 3D cameras, in combination with powerful GigE uEye CMOS cameras, are used.

Reliable and precise image

The integrated IDS industrial cameras ensure a reliable, precise image capture that is needed for Fizyr's software algorithms which provide more than 100 grasp poses each second, including the classification to handle objects differently. The software also performs quality controls and detects defects to prevent damaged items from being placed on a sorter – with the help of IDS cameras acting as the ‘sharp eyes’ of the automatic system.

Thus, these algorithms are able to provide all relevant information regarding segmentation, as well as about classification of type of parcel (including box, bag, envelope/flat, tube, cylinder, deformable etc.). The system recognises outliers or non-conveyables (ie, damaged goods), best possible grasp poses in 6 DoF (degrees of freedom) and multiple ordered poses per object. It allows sensors or robots to deal with closely stacked or overlapping objects, highly reflective items and apparel in polybags, white-on-white and black-on-black flats, as well as transparent objects.

Since the Stereo Vision quality directly depends on the scene’s light condition and object surface textures, finding and calculating coordinates of corresponding points on less textured or reflecting surfaces is very difficult. To meet these high requirements, Fizyr’s first choice for customer solutions are camera models out of the IDS portfolio. Thereby the tasks are well divided: The uEye GigE CP camera is taking 2D images of objects and provides them as input for Fizyr's algorithms, which then proceed to classify the objects under the camera. The objects can be unknown and varying in shape, size, colour, material and stacking. 

Gripping proposal

The Ensenso camera then creates the point cloud maps, Fizyr's software combines it with the information from the 2D image and analyses the surface of the cloud for suitable grasp poses for the gripper (or multiple grippers), and proposes the best ones. A clear representation of surfaces for different materials is critical, as it is a key component of its algorithms, readily managed by the Ensenso 3D cameras from IDS, whhc are said to improve the classic Stereo Vision principle by additional techniques to achieve a higher quality depth information and more precise measurement results.

The specific camera models, as well as the number of cameras per system, depend on the individual use case of the customer. For a typical bin-picking solution with a cobot, one Ensenso N35 is used in combination with a GigE uEye CP, but there are clients that use one Ensenso X36 and a GigE uEye CP for bin picking, together with four Ensenso N35 cameras for stowing the item in other bins.

The recommended uEye CP stands for ‘Compact Power’ and is the tiny powerhouse for industrial applications of all kinds. It offers maximum functionality with extensive pixel pre-processing and, thanks to the internal 120 MB image memory, is described as ideally suited for multi-camera systems. The camera delivers data at full GigE speed and enables single-cable operation up to 100 metres via PoE (Power over Ethernet).

Power with sensitivity

 The selected UI-524CP-C-HQ Rev. 2 is a particularly powerful industrial camera with the e2v 1.3 megapixel CMOS sensor. This is one of the most sensitive sensors in the IDS product portfolio and, apart from the colour version, it is available in monochrome and as an NIR version. Besides outstanding sensitivity to light, the camera also has a range of other distinctive features, which makes the sensor extremely flexible, if requirements or ambient conditions change.

The compact and robust aluminum housing of the Ensenso N35 3D camera, with lockable GPIO connector for trigger and flash and GigE connector, has two monochrome CMOS sensors (Global Shutter, 1280 x 1024 pixels) and a projector. Via Power over Ethernet, a data transfer and power supply are possible with long cable lengths. Due to the integrated FlexView technology the N35 models are particularly suitable for 3D acquisition of still objects and for working distances up to 3,000 mm. 

The Ensenso X36 3D camera system consists of a projector unit, two GigE uEye cameras, either with 1.3 MP or 5 MP sensors (CMOS, monochrome), mounting brackets and adjustment angles, three lenses, as well as sync and patch cables to connect the camera with the projector unit. The FlexView technology ensures a better spatial resolution, as well as a very high robustness of the system for dark or reflecting surfaces.

No limitations

"Fizyr normally uses one camera per system, usually uEye cameras in combination with Ensenso N35 and X36, but there are no limitations,” underlines Herbert ten Have, CEO at Fizyr. “The most common use case for Fizyr so far is one uEye and one Ensenso per system.”

The different Ensenso models that can be used have one thing in common: the dispose of a light-intensive projector produces a high-contrast texture on the object surface by using a pattern mask, even under difficult light conditions. The projected texture supplements the weak or non-existent object surface structure. Therefore, this principle is also called ‘Projected Texture Stereo Vision’. The result is a more detailed disparity map, and more complete and homogeneous depth information of the scene.

Their extensive features qualify both Ensenso 3D camera models for the wide range of demanding applications in the supply chain and logistics industry, such as 3D object recognition, classification and localisation (eg, quality assurance, commissioning), logistics automation (eg, de-palletising), robot applications (eg, bin picking), capture of objects up to 8 m³, (eg, pallets) or automatic storage systems.

"Fizyr has integrated the Ensenso SDK in its software using a modern and fast wrapper,” ten Have explains. "A great advantage is the 2D/3D combination, which allows the image from the 2D camera to be placed over the 3D point cloud as an overlay.” This provides a more vivid impression of the scene. On the other hand, the camera image can be optimally adjusted to the robot coordinate system by means of ‘hand-eye calibration’, in order to ensure a target-oriented gripping.

Increased demand for larger packages 

The strong growth of e-commerce has increased the demand for larger packages. At the same time, the constraints imposed by the global pandemic have led both to a reduction in the available labour force and to an urgent need for physical distance in companies. To meet these challenges, companies are increasingly turning to robotic automation, in which artificial intelligence and machine vision play a crucial role. Also, this reduces costs and increases occupational safety.

The demand for high-quality modules for state-of-the-art automated picking cells is growing worldwide. System integrators' requirements for software and hardware components are increasing in equal measure. The Fizyr-IDS solution is billed as offering the following advantages:

·         As a plug-and-play solution, it enables easy integration into any system

·         It is hardware-agnostic and thus enables customers to flexibly adapt their order picking system to their needs at any time

·         It includes a robust, precise, flexible and easy-to-use 3D vision system that provides optimal object information at all times.

All of these, it is stated, are “indispensable components” of a future-proof automated ‘Good-To-Person’ system.

 

The need for robotic automation has never been more urgent.
The need for robotic automation has never been more urgent.
Ensenso 3D camera allows robots individual handling of unknown objects.
Ensenso 3D camera allows robots individual handling of unknown objects.
The system robustly segments unknown parcels, even under harsh lighting conditions - see above and image below.
The system robustly segments unknown parcels, even under harsh lighting conditions - see above and image below.

IDS Imaging Development Systems

Landmark House
Station Road
Hook
RG27 9HA
UNITED KINGDOM

+44 (0)1256 962910

uksales@ids-imaging.com

www.ids-imaging.com

More from IDS Imaging Development Systems

Brushing up on best practice

Posted 1 week ago

Handle with care

Posted 1 month ago

IDS camera has Eye on the main prize

Posted 1 month ago

Robots on their own

Posted 1 month ago

All-in-one embedded vision platform with new tools and functions

Posted 2 months ago

Evolution in the press shop

Posted 2 months ago

From smallest housed camera with C-mount over 10GigE to AI

Posted 3 months ago

High order intake and promising new products

Posted 4 months ago

Triple vision in quest for excellence

Posted 4 months ago

IDS expands Ensenso 3D camera portfolio

Posted 5 months ago

Drive-through control

Posted 5 months ago

Time to take your pick

Posted 6 months ago

Eyeing up the new uEye sensors

Posted 6 months ago

Engage with image processing experts to ‘Capture the Light’

Posted 7 months ago

Eyes on the uEye XLE models

Posted 7 months ago

It takes a lot of bottle to fight the pandemic

Posted 7 months ago

Precision vision aids productivity

Posted 8 months ago

Making space for additive manufacturing

Posted 10 months ago

New industrial vision online channel sees the light of day

Posted 11 months ago

Affordable, compact and powerful new camera family

Posted 1 year ago

New C-Mount lenses bring focus for machine and system builders

Posted 1 year ago

IDS extends Ensenso N3D camera series

Posted 1 year ago

5 megapixels polarisation cameras reveal hidden details

Posted 1 year ago

Learn how to create an intelligent camera

Posted 1 year ago

IDS NXT ocean - unique all-in-one AI-based image processing

Posted 2 years ago

IDS launches industrial cameras with 20 MP Sony sensor

Posted 2 years ago

Modular uEye ACP for developing customised industrial cameras

Posted 2 years ago

Ensenso XR 3D camera family features integrated data processing

Posted 2 years ago

Board-level cameras with liquid lenses and autofocus

Posted 2 years ago

IDS launches more than 100 new models of USB3 Vision camera

Posted 2 years ago

On a voyage of discovery with IDS cameras

Posted 2 years ago

Vision app-based cameras now available from IDS

Posted 2 years ago

Registration opens for IDS UK Industrial Vision Days

Posted 2 years ago

uEye CP camera with high-resolution 12 MP rolling shutter sensor

Posted 2 years ago

Sony sensor: excellent image quality under difficult conditions

Posted 2 years ago

Board-level cameras feature liquid lenses and Active Focus

Posted 2 years ago

IDS achieves sales increase well above industry average

Posted 2 years ago

Self-programmed camera functions meet GenICam standard

Posted 2 years ago

3D vision system creates data twice as fast

Posted 3 years ago

IDS NXT rio and rome industrial cameras are AI-ready

Posted 3 years ago

IDS unveils Ensenso XR 3D camera with onboard processing

Posted 3 years ago

IDS unveils 10 GigE uEye FX high-speed camera with PoE

Posted 3 years ago

IDS strengthens presence in South Korea

Posted 3 years ago

IDS 3D vision system helps robots assemble flat-pack furniture

Posted 3 years ago

IDS at the world's leading trade fair for machine vision

Posted 3 years ago

New USB 3.1 board-level cameras with liquid lens control

Posted 3 years ago

IDS delivers its millionth camera

Posted 3 years ago

GigE Vision camera firmware 1.5 with new trigger modes

Posted 3 years ago

IDS NXT vegas model with colour sensor and further developments

Posted 3 years ago

3D camera system from IDS wins VSD Award

Posted 3 years ago

App-based industrial cameras now with colour sensors

Posted 3 years ago

EnsensoSDK 2.2 makes 3D image processing even more convenient

Posted 3 years ago

IDS introduces app-based vision sensors and cameras

Posted 3 years ago

See new app-based cameras and 3D cameras at UKIVA event

Posted 3 years ago

uEye Python interface for interactive programming of IDS cameras

Posted 4 years ago

How to simplify the prototyping of embedded vision systems

Posted 4 years ago

Robust 3D camera with IP65/67 for harsh ambient conditions

Posted 4 years ago

Versatile industrial camera with GigE or USB 3.1 Type-C port

Posted 4 years ago

New Gigabit Ethernet industrial cameras with CMOS sensors

Posted 4 years ago

IDS Vision Suite: easy evaluation, setup of GigE Vision cameras

Posted 4 years ago

IDS: successful in the vision market for two decades

Posted 4 years ago

Robust IP65/67 GigE uEye FA industrial cameras with PoE

Posted 4 years ago

IDS presents GigE uEye CP Gigabit Ethernet machine vision camera

Posted 4 years ago

New USB 3.1 Gen 1 cameras with the PYTHON 480 sensor

Posted 4 years ago

3D camera system is easier to implement and more versatile

Posted 4 years ago

IDS highlights embedded vision at UKIVA Conference

Posted 4 years ago

New USB 3.0 industrial camera models with Sony sensors from IDS

Posted 4 years ago

IDS enhances presence in Great Britain and Ireland

Posted 4 years ago

GigE industrial cameras from IDS for use in harsh environments

Posted 5 years ago

IDS adds GigE vision interface support to industrial cameras

Posted 5 years ago

IDS shows new products and developments at VISION show 2016

Posted 5 years ago

First USB 3.1 Gen 1 machine vision cameras with Type-C connector

Posted 5 years ago

IDS Imaging Development Systems among the top 100

Posted 5 years ago

USB 3.0 industrial cameras with next-generation CMOS sensors

Posted 5 years ago

USB 3 uEye XC machine vision camera wins Red Dot Award

Posted 5 years ago

New TechTip: use of sequencer to vary image acquisition settings

Posted 5 years ago

New version 4.80 of the IDS Software Suite available

Posted 5 years ago

New IDS USB3 cameras with fast Python CMOS sensors

Posted 5 years ago

USB 3.0 single-board camera with 18 MP Aptina colour sensor

Posted 5 years ago

Adaptive Vision machine vision software available from IDS in UK

Posted 6 years ago

GigE and USB 3.0 cameras with Sony IMX249 sensor from IDS

Posted 6 years ago

50m cable lengths for USB 3.0 cameras approach GigE capabilities

Posted 6 years ago

High-performance CMOSIS sensors for IDS USB3 industrial cameras

Posted 6 years ago

New lenses extend applications for IDS uEye XC 13 MP USB3 camera

Posted 6 years ago

New robust 3D stereo cameras are sealed to IP65/67

Posted 6 years ago

Daniel Seiler joins IDS management board

Posted 6 years ago

IDS and Stemmer Imaging to end collaboration

Posted 6 years ago

UVC version of USB 3 autofocus industrial camera released by IDS

Posted 6 years ago

IDS to host Future Vision Day in UK on 17 June 2015

Posted 6 years ago

Platform-independent, board-level USB industrial cameras

Posted 6 years ago

IDS pushes forward with CMOS industrial cameras

Posted 6 years ago

New technical note for multicast configuring of IDS GigE cameras

Posted 6 years ago

High-performance CMOS sensors for IDS USB 3.0 CP cameras

Posted 7 years ago

Technical note on new methodology for high-speed imaging

Posted 7 years ago

IDS to launch two new USB 3.0 cameras at VISION 2014

Posted 7 years ago

GigE 3D stereo camera for 3D and robot vision applications

Posted 7 years ago

IDS extends its USB 3.0 camera portfolio with three new families

Posted 7 years ago

GigE industrial cameras with 2.2 and 4.2 MP CMOS sensors

Posted 7 years ago

Ensenso 3D camera makes bin picking easy and cost efficient

Posted 7 years ago

More news

Login / Sign up