Scientists in Japan have developed a new 3D object detection system that overcomes the challenges unmanned vehicles encounter in adverse weather conditions. Professor Hiroyuki Tomiyama from Ritsumeikan University, Japan, led a research team to create DPPFA−Net (Dynamic Point-Pixel Feature Alignment Network), which makes multi-modal 3D object detection more accurate and robust, the team claimed. According […]
Scientists in Japan have developed a new 3D object detection system that overcomes the challenges unmanned vehicles encounter in adverse weather conditions.
Professor Hiroyuki Tomiyama from Ritsumeikan University, Japan, led a research team to create DPPFA−Net (Dynamic Point-Pixel Feature Alignment Network), which makes multi-modal 3D object detection more accurate and robust, the team claimed.
According to the study, published in the IEEE Internet of Things Journal, most 3D object detection methods use LiDAR sensors to scan and measure the distances of objects and surfaces around the source, however this data alone can lead to errors due to the high sensitivity of LiDAR to noise, especially in adverse weather conditions like rainfall.
The DPPFA−Net model comprises multiple instances of three novel modules: the Memory-based Point-Pixel Fusion (MPPF) module; the Deformable Point-Pixel Fusion (DPPF) module, and the Semantic Alignment Evaluator (SAE) module.
The MPPF module is tasked with performing interactions between intra-modal features (2D with 2D and 3D with 3D) and cross-modal features (2D with 3D).
The use of the 2D image as a memory bank is said to reduce the difficulty in network learning and makes the system more robust against noise in 3D point clouds. Moreover, it promotes the use of more comprehensive and discriminative features.
In contrast, the DPPF module performs interactions only at pixels in key positions, which are determined via a smart sampling strategy. This allows for feature fusion in high resolutions at a low computational complexity.
Finally, the SAE module helps ensure semantic alignment between both data representations during the fusion process, which mitigates the issue of feature ambiguity.
DPPFA-Net explained
The team evaluated DPPFA−Net against the top performers for the widely used KITTI Vision Benchmark.
The network achieved precision improvements as high as 7% under different noise conditions. The team also created a new noisy dataset by introducing artificial multi-modal noise in the form of rainfall to the KITTI dataset.
The results show that the proposed network performed better than existing models not only in the face of severe occlusions but also under various levels of adverse weather conditions.
“Our extensive experiments on the KITTI dataset and challenging multi-modal noisy cases reveal that DPPFA-Net reaches a new state-of-the-art,” said Tomiyama.
“Our study could facilitate a better understanding and adaptation of robots to their working environments, allowing a more precise perception of small targets,” he added. “Such advancements will help improve the capabilities of robots in various applications.”
Last year, we reported on a research team at the University of Cambridge which had designed a low cost energy efficient robotic hand capable of grasping objects as delicate as an egg, by using the movement in its wrist and the feeling in its skin.
