Research on real-time distance measurement of mobile eye tracking system based on neural network

Research on real-time distance measurement of mobile eye tracking system based on neural network

Table of Contents


With the development and application of eye-tracking technology, mobile eye-tracking systems have become more widely used due to their safety and portability. We combine eye-tracking systems with real-time object detection using machine learning. We propose a method of wearing an eye tracker in daily life to obtain the distance between the eye-tracking system and the gaze target in real-time. During the visual interaction of the eye-tracking system, in order to obtain the distance from the eyeball fixation target to the eyeball in real-time, the world camera of the mobile eye-tracking system pupillabs first collects the position and scale information of the detected target image in real-time and uses camera calibration principle, pinhole camera model and camera distortion model establish a ranging equation, and then the feasibility of the real-time ranging equations verified through a specified distance experiment. The total average relative error after de-distortion at the position of 50cm-75cm is reduced to 1.25%, and the highest accuracy-0.9182cm distance measurement can be achieved within the effective distance.

  • Author Keywords

    • the mobile eye-tracking system,
    • neural network,
    • pinhole camera,
    • distance measurement
  • IEEE Keywords

    • Cameras,
    • Gaze tracking,
    • Mathematical model,
    • Distortion,
    • Object detection,
    • Real-time systems,
    • Distance measurement
  • Controlled Indexing

    • calibration,
    • cameras,
    • distance measurement,
    • eye,
    • gaze tracking,
    • image sensors,
    • learning (artificial intelligence),
    • neural nets,
    • object detection
  • Non-Controlled Indexing

    • real-time distance measurement,
    • eye-tracking technology,
    • mobile eye-tracking systems,
    • real-time object detection,
    • eye tracker,
    • mobile eye-tracking system pupil labs,
    • pinhole camera model,
    • camera distortion model,
    • neural network


The eye-tracking system is mainly used to study human behavior and understand the cognitive process[1]. It is mainly divided into two types: desktop fixed and mobile eye-tracking systems. The advent and development of wearable mobile eye-tracking systems have opened up new directions for tracking and analyzing human visual behavior during everyday activities. With the rapid development of human-computer visual interaction needs, the current mainstream commercial mobile eye-tracking systems mainly include Tobii glasses II developed by the Swedish Tobii company[2], the same mobile eye tracker designed and manufactured by the German SMI company[3], and Pupil labs developed by the Eye-tracking laboratory in Berlin, Germany[4], these eye-tracking system can obtain the angle information of the line of sight in the three-dimensional space through different line-of-sight estimation algorithms and the accuracy of the line-of-sight tracking can reach o0.5. Eye distance has become another requirement for the development of eye-tracking systems.

We use the eye tracker developed by the German Eye Tracking Laboratory-Pupil labs. The camera parameters are shown in Table I. This eye-tracking system is based on binocular recognition. The main configuration includes two 850nm band near-infrared cameras with eye recognition and a world camera that captures the wearer’s field of view, of which the near-infrared camera has a frame rate of 120 frames per second and an image resolution of 320 × 240 pixels; the world camera corresponds to a variety of sampling frequencies, 30hz @ 1080p, 60hz @ 720p, 120hz @ 320 × 240 pixels, etc. Here, in order to get a wider field of view of the wearer, we choose the highest image resolution in the world camera.


Based on the research of head-mounted eye-tracking system, we propose a measurement method that uses the combination of the TensorFlow API framework target detection and eye-tracking system to obtain the distance between the eye-tracking point and the wearer in real-time. Auxiliary equipment is required, and distance measurement has better accuracy. The system is first assembled with a TensorFlow-based object detection plug-in, followed by camera calibration, the distortion model and image distance equation are established, and finally the distance measurement is completed through six sets of distance experiments. The experimental results show that when the target detection framework is incorporated into the pupil labs, the total average accuracy of the target detection is 91.85%, the total average relative error after de-distortion is reduced to 1.25%, and the highest accuracy of the ranging is -0.9182 cm.

About KSRA

The Kavian Scientific Research Association (KSRA) is a non-profit research organization to provide research / educational services in December 2013. The members of the community had formed a virtual group on the Viber social network. The core of the Kavian Scientific Association was formed with these members as founders. These individuals, led by Professor Siavosh Kaviani, decided to launch a scientific / research association with an emphasis on education.

KSRA research association, as a non-profit research firm, is committed to providing research services in the field of knowledge. The main beneficiaries of this association are public or private knowledge-based companies, students, researchers, researchers, professors, universities, and industrial and semi-industrial centers around the world.

Our main services Based on Education for all Spectrum people in the world. We want to make an integration between researches and educations. We believe education is the main right of Human beings. So our services should be concentrated on inclusive education.

The KSRA team partners with local under-served communities around the world to improve the access to and quality of knowledge based on education, amplify and augment learning programs where they exist, and create new opportunities for e-learning where traditional education systems are lacking or non-existent.

FULL Paper PDF file:

Research on real-time-distance measurement of mobile eye-tracking system based on neural network



L. Hu and J. Gao,




Research on real-time distance measurement of a mobile eye-tracking system based on neural network

Publish in

2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 2020, pp. . 1561-1565,



PDF reference and original file: Click here

+ posts

Somayeh Nosrati was born in 1982 in Tehran. She holds a Master's degree in artificial intelligence from Khatam University of Tehran.

Website | + posts

Professor Siavosh Kaviani was born in 1961 in Tehran. He had a professorship. He holds a Ph.D. in Software Engineering from the QL University of Software Development Methodology and an honorary Ph.D. from the University of Chelsea.

Website | + posts

Nasim Gazerani was born in 1983 in Arak. She holds a Master's degree in Software Engineering from UM University of Malaysia.