Image based rainfall amount estimation for auto-wiping of vehicles

With rapid development of deep learning, a lot of computer vision tasks, such as object detection and semantic segmentation, have been applied to various Advanced Driver Assistance Systems. However, few computer vision solutions to estimate rainfall amount have been developed so far. So, we propose a rainfall amount estimation method based on deep learning and computer vision. The proposed method mainly consists of two steps. The first step is raindrop segmentation, and the second step is rainfall amount estimation. The raindrop segmentation is specifically based on three techniques: A relational ASPP to explore the correlation between raindrop features, a height attention module to consider the features that vary depending on raindrop locations, and a masking loss to further improve the performance of raindrop segmentation. Second, using the segmented raindrops, we present a rainfall amount estimation algorithm for auto-wiping. We experimentally achieved acceptable raindrop segmentation performance, i.e., mean IoU (mIoU) score of 70.6% that is much higher than other algorithms. This verifies that the proposed network is good at segmenting raindrops on a windshield. And, we demonstrate that the proposed rainfall amount estimation scheme provides sufficiently high accuracy of about 93%. In addition, we have built a rainy driving dataset for computer vision-based auto-wiping purpose and published it publicly on https://github.com/jjh930910/raindrop-segmentation.