Accurate Eye Pupil Localization Using Heterogeneous CNN Models

Eye pupil localization is one of the indispensable technologies in various computer vision applications such as virtual reality and augmented reality. In general, the algorithm consists of finding the approximate eye region and finding the pupil position by extracting the semantic feature from each eye region. However, the performance is affected not only by illumination and image resolution but also by glasses wear. Therefore, this paper proposes an eye pupil localization algorithm which is robust against the above disturbance conditions and also has high accuracy using heterogeneous CNN models. First, faces in the image and landmarks in the face(s) are detected sequentially, and the eye region is determined based on the landmarks. Especially, if glasses are present, the glasses are removed by GAN to find the correct eye region. Next, the pupil region is segmented using fully convolutional networks. Finally, the position of the segmented pupil is calculated. Experimental results show that the proposed algorithm outperforms the state-of-the-art algorithms for public databases such as BioID and GI4E.