Eye-tracking has become a significant tool in various areas, incorporating interaction of humans with a computer, computer vision, psychology, and medical diagnostics. Various protocols have been utilized to track the gaze. Although, some may not be accurate in the real world, while others may need explicit custom calibration, which can cause problems. Few of these techniques are associated with low image grades and varying lighting situations. The latest success and popularity of deep studying has dramatically increased the effectiveness of eye-tracking. The convenience of huge datasets additionally improves the function of deep learning-depend techniques. This technical paper introduces the latest deep studying-depend gaze assessment technology with a concentration on Fully Convolution Neural Networks (FCNN). This technical paper also gives an overview of other machine-based eye assessment methods. This research objective to enable the research population to generate significant and useful horizons that can improve the structure and growth of better, more effective deep learning depends on eye-tracking methods. This paper also gives information on different pertained models, network structures, and open origin datasets to help train deep learning models.
Reference
K. Wang, S. Wang, and Q. Ji, “Deep eye fixation map learning for calibration-free eye gaze tracking,” in Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, New York, NY, 2016. doi: https://doi.org/10.1145/2857491.2857515
R. A. Naqvi, M. Arsalan, G. Batchuluun, H. S. Yoon, and K. R. Park, “Deep learning-based gaze detection system for automobile drivers using a NIR camera sensor,” Sensors, vol. 18, no. 2, p. 456, 2018. doi: https://doi.org/10.3390/s18020456
S. Robertson, G. Penn, and Y. Wang. (2019) Exploring spectro-temporal features in end-to-end convolutional neural networks. [Online]. Available: https://arxiv.org/pdf/1901.00072.pdf
Y. Kim, Y. Jernite, D. Sontag, and A. M. Rush, “Character-aware neural language models,” in Thirtieth AAAI Conference on Artificial Intelligence, Menlo Park, CA, 2016.
W. Wang and J. Shen, “Deep visual attention prediction,” IEEE Transactions on Image Processing, vol. 27, no. 5, pp. 2368–2378, 2017. doi: https://doi.org/10.1109/TIP.2017.2787612
A. Tsukada, M. Shino, M. Devyver, and T. Kanade, “Illumination-free gaze estimation method for first-person vision wearable device,” in International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 2011. doi: https://doi.org/10.1109/ICCVW.2011.6130505
D. W. Hansen and Q. Ji, “In the eye of the beholder: A survey of models for eyes and gaze,” Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 3, pp. 478–500, 2009. doi: https://doi.org/10.1109/TPAMI.2009.30
C. N. M. Locsin and R. J. Ferolin, “Neural networks application for water distribution demand-driven decision support system,” Journal of Advances in Technology and Engineering Research, vol. 4, no. 4, pp. 160–175, 2018. doi: https://doi.org/10.20474/jater-4.4.3
H. Chennamma and X. Yuan. (2013) A survey on eye-gaze tracking techniques. [Online]. Available: https://arxiv.org/abs/1312.6410
M. Quazi, “An overview of laser welding of highstrength steels for automotive application,” International Journal of Technology and Engineering Studies, vol. 6, no. 1, pp. 23–40, 2020. doi: https://doi.org/10.20469/ijtes.6.10004-1
X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “Appearance-based gaze estimation in the wild,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, 2015.
S. F. Haider, M. Quazi, J. Bhatti, M. N. Bashir, and I. Ali, “Effect of Shielded Metal Arc Welding (SMAW) parame-ters on mechanical properties of low-carbon, mild and stainless-steel welded joints: A review,” Journal of Advances in Technology and Engineering Research, vol. 5, no. 5, pp. 191–198, 2019. doi: https://doi.org/10.20474/jater-5.5.1
C. Zhang, R. Yao, and J. Cai, “Efficient eye typing with 9-direction gaze estimation,” Multimedia Tools and Applications, vol. 77, no. 15, pp. 19 679–19 696, 2018. doi: https://doi.org/10.1007/s11042-017-5426-y
N. Z. T. Abdulnabi and O. Altun, “Batch size for training convolutional neural networks for sentence classification,” Journal of Advances in Technology and Engineering Studies, vol. 2, no. 5, pp. 156–163, 2016. doi: https://doi.org/10.20474/jater-2.5.3
I. Ali, N. Lin, M. Quazi, M. N. Bashir, H. Sadiq, and F. Sharaf, “Investigating the wear characteristics of metal matrix composite coating deposited on AA5083 Al-alloy by laser surface engineering technique,” North American Academic Research, vol. 3, no. 1, pp. 138–146, 2019. doi: https://doi.org/10.5281/zenodo.3626448
Y. Sugano, Y. Matsushita, Y. Sato, and H. Koike,“Appearance-based gaze estimation with online calibration from mouse operations,” IEEE Transactions on Human-Machine Systems, vol. 45, no. 6, pp. 750–760, 2015. doi: https://doi.org/10.1109/THMS.2015.2400434
X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “It’swritten all over your face: Full-face appearance-based gaze estimation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, 2017.
M. Gul, N. Zulkifli, M. Kalam, H. Masjuki, M. Mujtaba, S. Yousuf, M. N. Bashir, W. Ahmed, M. Yusoff, S. Noor et al., “RSM and artificial neural networking based production optimization of sustainable cotton bio-lubricant and evaluation of its lubricity & tribological properties,” Energy Reports, vol. 7, pp. 830–839, 2021. doi: https://doi.org/10.1016/j.egyr.2021.01.033
M. M. Quazi, M. Ishak, A. Arslan, M. Nasir Bashir,and I. Ali, “Scratch adhesion and wear failure characteristics of pvd multilayer crti/crtin thin film ceramic coating deposited on AA7075-T6 aerospace alloy,” Journal of Adhesion Science and Technology, vol. 32, no. 6, pp. 625–641, 2018. doi: https://doi.org/10.1080/01694243.2017.1373988
Y. Wang, T. Shen, G. Yuan, J. Bian, and X. Fu,“Appearance-based gaze estimation using deep features and random forest regression,” Knowledge-Based Systems, vol. 110, pp. 293–301, 2016. doi: https://doi.org/10.1016/j.knosys.2016.07.038
M. M. Quazi, M. Ishak, A. Arslan, M. Fazal, F. Yusof, B. Sazzad, M. N. Bashir, and M. Jamshaid, “Mechanical and tribological performance of a hybrid MMC coating deposited on Al-17Si piston alloy by laser composite surfacing technique,” RSC Advances, vol. 8, no. 13, pp. 6858–6869, 2018. doi:https://doi.org/10.1039/C7RA08191J
A. George and A. Routray, “Real-time eye gaze direction classification using convolutional neural network,” in International Conference on Signal Processing and Communications (SPCOM), Bangalore, India, 2016.
A. Kar and P. Corcoran, “Performance evaluationstrategies for eye gaze estimation systems with quantitative metrics and visualizations,” Sensors, vol. 18, no. 9, pp. 1–35, 2018. doi: https://doi.org/10.3390/s18093151
R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, “Convolutional neural networks: An overview and application in radiology,” Insights into Imaging, vol. 9, no. 4, pp. 611–629, 2018. doi:https://doi.org/10.1007/s13244-018-0639-9
T. Huff, N. Mahabadi, and P. Tadi, Neuroanatomy,Visual Cortex. Treasure Island, FL, StatPearls Publishing, 2020.
S. Wakeel, S. Ahmad, S. Bingol, M. N. Bashir, T. C.Paçal, and Z. A. Khan, “Supplier selection for high temperature die attach by hybrid entropy-range of value MCDM technique: A semiconductor industry,” in 21st International Conference on Electronic Packaging Technology (ICEPT), Guangzhou, China, 2020. doi: https://doi.org/10.1109/ICEPT50128.2020.9202994
B. Ramsundar and R. B. Zadeh, TensorFlow fordeep learning: From linear regression to reinforcement learning. Sebastopol, CA: O’Reilly Media, Inc, 2018.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105, 2012.
S. Vora, A. Rangesh, and M. M. Trivedi, “On generalizing driver gaze zone estimation using convolutional neural networks,” in Intelligent Vehicles Symposium (IV), Los Angeles, CA, 2017. doi:https://doi.org/10.1109/IVS.2017.7995822
S. Rustagi, A. Garg, P. R. Anand, R. Kumar, Y. Kumar, and R. R. Shah, “Touchless typing using head movement-based gestures,” in IEEE Sixth International Conference on Multimedia Big Data (BigMM), New Delhi, India, 2020. doi: https://doi.org/10.1109/BigMM50055.2020.00025
G. Bradski, “The openCV library,” Dr. Dobb’s Journal: Software Tools for the Professional Programmer, vol. 25, no. 11, pp. 120–123, 2000.
S. Wakeel, S. Bingol, S. Ahmad, M. N. Bashir, M. S. M. M. Emamat, Z. Ding, and F. Hussain, “A new hybrid LGPMBWM-PIV method for automotive material selection,” Informatica, vol. 45, no. 1, pp.105–115, 2021.
Y. Lee, C. Shin, A. Plopski, Y. Itoh, T. Piumsomboon, A. Dey, G. Lee, S. Kim, and M. Billinghurst, “Estimating gaze depth using multi-layer perceptron,” in International Symposium on Ubiquitous Virtual Reality (ISUVR), Nara, Japan, 2017. doi: https://doi.org/10.1109/ISUVR.2017.13
L. L. Di Stasi, R. Renner, A. Catena, J. J. Cañas, B. M. Velichkovsky, and S. Pannasch, “Towards a driver fatigue test based on the saccadic main sequence: A partial validation by subjective report data,” Transportation Research Part C: Emerging Technologies, vol. 21, no. 1, pp. 122–133, 2012.
S. D. Goldinger and M. H. Papesh, “Pupil dilationreflects the creation and retrieval of memories,” Current Directions in Psychological Science, vol. 21, no. 2, pp. 90–95, 2012. doi: https://doi.org/10.1177/0963721412436811
S. Vora, A. Rangesh, and M. M. Trivedi, “Drivergaze zone estimation using convolutional neural networks: A general framework and ablative analysis,”IEEE Transactions on Intelligent Vehicles, vol. 3, no. 3, pp. 254–265, 2018. doi: https://doi.org/10.1109/TIV.2018.2843120
C. Shin, G. Lee, Y. Kim, J. Hong, S.-H. Hong, H. Kang, and Y. Lee, “Evaluation of gaze depth estimation using a wearable binocular eye tracker and machine learning,” Journal of The Korea Computer Graphics Society, vol. 24, no. 1, pp. 19–26, 2018.
M. Trokielewicz, A. Czajka, and P. Maciejewicz,“Post-mortem iris recognition with deep-learning-based image segmentation,” Image and Vision Computing, vol. 94, pp. 1–34, 2020. doi: https://doi.org/10.1016/j.imavis.2019.103866
S. Jia, A. Seccia, P. Antonenko, R. Lamb, A. Keil,M. Schneps, M. Pomplun et al., “Biometric recognition through eye movements using a recurrent neural network,” in International Conference on Big Knowledge (ICBK), Singapore, 2018. doi: https://doi.org/10.1109/ICBK.2018.00016
Z. Liang, F. Tan, and Z. Chi, “Video-based biometric identification using eye tracking technique,” in International Conference on Signal Processing, Communication and Computing (ICSPCC 2012), Hong Kong, China, 2012. doi: https://doi.org/10.1109/ICSPCC.2012.6335584
O. Ferhat and F. Vilariño, “Low cost eye tracking: The current panorama,” Computational Intelligence and Neuroscience, vol. 2016, pp. 1–14, 2016. doi:https://doi.org/10.1155/2016/8680541
W. Zhu and H. Deng, “Monocular free-head 3dgaze tracking with deep learning and geometry constraints,” in Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017.
Y. Sugano, Y. Matsushita, and Y. Sato, “Learning-by-synthesis for appearance-based 3d gaze estimation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014, pp. 1821–1828.
D. Masko, “Calibration in eye tracking using transfer learning,” KTH Royal Institute of Technology, Stockholm, Sweden, M.S. thesis, 2017.
C. Palmero, J. Selva, M. A. Bagheri, and S. Escalera. (2018) Recurrent cnn for 3d gaze estimation using appearance and shape cues. [Online]. Available: https://arxiv.org/abs/1805.03064
J. Niemann and C. Fussenecker, “Analysis of eye tracking usage in different domains and possible applications in the engineering environment,” in International Conference on Quality and Innovation in Engineering and Management, Napoca, Romania, 2014.
X. Zhou, H. Cai, Z. Shao, H. Yu, and H. Liu, “3deye model-based gaze estimation from a depth sensor,” in International Conference on Robotics and Biomimetics (ROBIO), Qingdao, China, 2016. doi: https://doi.org/10.1109/ROBIO.2016.7866350
M. Gul, M. Kalam, M. Mujtaba, S. Alam, M. N. Bashir, I. Javed, U. Aziz, M. R. Farid, M. T. Hassan, and S. Iqbal, “Multi-objective-optimization of process parameters of industrial-gas-turbine fueled with natural gas by using Grey-Taguchi and ANN methods for better performance,” Energy Reports, vol. 6, pp. 2394–2402, 2020. doi: https://doi.org/10.1016/j.egyr.2020.08.002
B. A. Smith, Q. Yin, S. K. Feiner, and S. K. Nayar, “Gaze locking: passive eye contact detection for human-object interaction,” in Proceedings of the 26th annual ACM symposium on User Interface Software and Technology, New York, NY, 2013. doi: https://doi.org/10.1145/2501988.2501994
I. Ali, M. Quazi, E. Zalnezhad, A. A. Sarhan, N. L. Sukiman, and M. Ishak, “Hard anodizing of aerospace AA7075-T6 aluminum alloy for improving surface properties,” Transactions of the Indian Institute of Metals, vol. 72, no. 10, pp.2773–2781, 2019. doi: https://doi.org/10.1007/s12666-019-01754-5
Vision Aware. (2020) Driving with low vision. [Online]. Available: https://bit.ly/3myBojj
Y. LeCun. (2013) LeNet-5, convolutional neural networks. [Online]. Available: https://bit.ly/3BwOphx
C. Ma, K.-A. Choi, B.-D. Choi, and S.-J. Ko, “Robust remote gaze estimation method based on multiple geometric transforms,” Optical Engineering, vol. 54, no. 8, p. 083103, 2015. doi: https://doi.org/10.1117/1.OE.54.8.083103
C.-C. Lai, S.-W. Shih, and Y.-P. Hung, “Hybrid method for 3-D gaze tracking using glint and contour features,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 1, pp.24–37, 2014.
R. Newman, Y. Matsumoto, S. Rougeaux, and A. Zelinsky, “Real-time stereo tracking for headpose and gaze estimation,” in Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580), Grenoble, France, 2000. doi: https://doi.org/10.1109/AFGR.2000.840622
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, 2015.
P. Paysan, R. Knothe, B. Amberg, S. Romdhani,and T. Vetter, “A 3d face model for pose and illumination invariant face recognition,” in 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, Genova, Italy. Ieee, 2009, pp. 296–301.
O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deepface recognition,” in Proceedings of British Machine Vision Association, Swansea, UK, 2015. doi: https://dx.doi.org/10.5244/C.29.41
A. Galante and P. Menezes, “A gaze-based interaction system for people with cerebral palsy,” Procedia Technology, vol. 5, pp. 895–902, 2012. doi: https://doi.org/10.1016/j.protcy.2012.09.099
P. Blignaut, “Mapping the pupil-glint vector to gaze coordinates in a simple video-based eye tracker,” Journal of Eye Movement Research, vol. 7, no. 1, pp.1–11, 2014. doi: https://doi.org/10.16910/jemr.7.1.4
J. Zhu and J. Yang, “Subpixel eye gaze tracking,” in Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition, Washington, DC, WA, 2002. doi: https://doi.org/10.1109/AFGR.2002.1004144
Z. Zhu, Q. Ji, and K. P. Bennett, “Nonlinear eyegaze mapping function estimation via support vector regression,” in 18th International Conference on Pattern Recognition (ICPR’06), Hong Kong, China, vol. 1, 2006. doi: https://doi.org/10.1109/ICPR.2006.864
J. Kim, M. Stengel, A. Majercik, S. De Mello, D. Dunn, S. Laine, M. McGuire, and D. Luebke, “Nvgaze: An anatomically-informed dataset for low-latency, near-eye gaze estimation,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY, 2019. doi:https://doi.org/10.1145/3290605.3300780
Y.-G. Shin, K.-A. Choi, S.-T. Kim, C.-H. Yoo, and S.-J. Ko, “A novel 2-D mapping-based remote eye gaze tracking method using two IR light sources,” in IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, 2015. doi: https://doi.org/10.1109/ICCE.2015.7066375
J. M. Franchak, K. S. Kretch, K. C. Soska, and K. E. Adolph, “Head-mounted eye tracking: A new method to describe infant looking,” Child Development, vol. 82, no. 6, pp. 1738–1750, 2011. doi: https://doi.org/10.1111/j.1467-8624.2011.01670.x
K. Simonyan and A. Zisserman. (2014) Very deep convolutional networks for large-scale image recognition. [Online]. Available: https://arxiv.org/abs/1409.1556
To Cite this article
M. Raza, C. Wang, P. XUE, and S. Bano, “Research on eye pupil location and eye tracking by designing a Fully Conventional Neural Network (FCNN),” International Journal of Technology and Engineering Studies, vol. 6, no. 2, pp. 52 -68, 2020.