e-Vision: An AI-powered system for promoting the autonomy of visually impaired

Authors

  • Panagiotis Migkotzidis Centre for Research and Technology Hellas, Information Technologies Institute, MKlab, Thermi-Thessaloniki, 57001, Greece
  • Fotis Kalaganis Centre for Research and Technology Hellas, Information Technologies Institute, MKlab, Thermi-Thessaloniki, 57001, Greece
  • Kostas Georgiadis Centre for Research and Technology Hellas, Information Technologies Institute, MKlab, Thermi-Thessaloniki, 57001, Greece
  • Elisavet Chatzilari Centre for Research and Technology Hellas, Information Technologies Institute, MKlab, Thermi-Thessaloniki, 57001, Greece
  • George Pehlivanides Tetragon S.A., Karaiskaki 2B str, 54641 Thessaloniki, Greece
  • Spyros Tsafaras Tetragon S.A., Karaiskaki 2B str, 54641 Thessaloniki, Greece
  • Kostas Monastiridis Tetragon S.A., Karaiskaki 2B str, 54641 Thessaloniki, Greece
  • George Martinidis Aristotle University of Thessaloniki, School of Architecture, Thessaloniki, 54124, Greece
  • Spiros Nikolopoulos Centre for Research and Technology Hellas, Information Technologies Institute, MKlab, Thermi-Thessaloniki, 57001, Greece
  • Ioannis Kompatsiaris Centre for Research and Technology Hellas, Information Technologies Institute, MKlab, Thermi-Thessaloniki, 57001, Greece

DOI:

https://doi.org/10.6092/issn.2612-0496/10707

Keywords:

computer vision, assistive technologies, blind

Abstract

Computer vision-based assistive technology for the visually impaired is still a field of ongoing research. Its fundamental scope is to extend the frontiers of visually impaired by means of providing a greater degree of independence and autonomy in their daily living activities. Towards this direction, we present the “e-Vision”, a hybrid system that couples the convenience and the inherently seamless adoption of an external camera embedded within a pair of eyeglasses with the processing power of modern smartphone devices. The system consists of a pair of eyeglasses integrating a camera and a mobile application that encapsulates computer vision algorithms capable to enhance several daily living tasks for the visually impaired. The proposed system is a context-aware solution and builds upon three important day-to-day activities: visiting a super-market, going an outdoor walk and carrying out a work at a public service. Going one step further, the e-Vision also caters for social inclusion by providing social context and enhances overall experience by adopting soundscapes that allow users to perceive selected points of interest in an immersive acoustic way.

References

Auvray, Malika, Sylvain Hanneton, and J. Kevin O'Regan. "Learning to perceive with a visuo—auditory substitution system: localisation and object recognition with ‘The Voice’." Perception 36, no. 3 (2007): 416-430.

Bourbakis, Nikolaos, Robert Keefer, Dimitrios Dakopoulos, and Anna Esposito. "A multimodal interaction scheme between a blind user and the tyflos assistive prototype." In 2008 20th IEEE International Conference on Tools with Artificial Intelligence, vol. 2, pp. 487-494. IEEE, 2008.

Gold, D., and H. Simson. "Identifying the needs of people in Canada who are blind or visually impaired: Preliminary results of a nation-wide study." In International Congress Series, vol. 1282, pp. 139-142. Elsevier, 2005.

Brady, Erin, Meredith Ringel Morris, Yu Zhong, Samuel White, and Jeffrey P. Bigham. "Visual challenges in the everyday lives of blind people." In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 2117-2126. 2013.

Martin, Robert C. "Design principles and design patterns." Object Mentor 1, no. 34 (2000): 597.

Alonso, Fernando, José L. Fuertes, Ángel L. González, and Loïc Martínez. "User-interface modelling for blind users." In International Conference on Computers for Handicapped Persons, pp. 789-796. Springer, Berlin, Heidelberg, 2008.

Sierra, Javier Sánchez, and J. S. Togores. "Designing mobile apps for visually impaired and blind users." In The Fifth international conference on advances in computer-human interactions, pp. 47-52. 2012.

Olofsson, Stina. "Designing interfaces for the visually impaired: Contextual information and analysis of user needs." (2018).

Qin, Zheng, Zhaoning Zhang, Xiaotao Chen, Changjian Wang, and Yuxing Peng. "Fd-mobilenet: Improved mobilenet with a fast downsampling strategy." In 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 1363-1367. IEEE, 2018.

Deng, Jia, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. "Imagenet: A large-scale hierarchical image database." In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. IEEE, 2009.

Redmon, Joseph, Santosh Divvala, Ross Girshick, and Ali Farhadi. "You only look once: Unified, real-time object detection." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788. 2016.

Kuznetsova, Alina, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali et al. "The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale." arXiv preprint arXiv:1811.00982 (2018).

Sun, Yi, Xiaogang Wang, and Xiaoou Tang. "Deep convolutional network cascade for facial point detection." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3476-3483. 2013.

Jourabloo, Amin, and Xiaoming Liu. "Pose-invariant face alignment via CNN-based dense 3D model fitting." International Journal of Computer Vision 124, no. 2 (2017): 187-203.

Bartz, Christian, Haojin Yang, and Christoph Meinel. "STN-OCR: A single neural network for text detection and text recognition." arXiv preprint arXiv:1707.08831 (2017).

Walker, Jake, Yasuhisa Fujii, and Ashok C. Popat. "A web-based ocr service for documents." In Proceedings of the 13th IAPR International Workshop on Document Analysis Systems (DAS), Vienna, Austria, vol. 1. 2018.

Anonymized for double blind review "A Computer Vision System Supporting Blind People-The Supermarket Case." In International Conference on Computer Vision Systems, pp. 305-315. Springer, Cham, 2019.

Mirandola, Daniela, Marco Monaci, Guido Miccinesi, Alessia Vannuzzi, Eleonora Sgambati, Mirko Manetti, and Mirca Marini. "Psychological well-being and quality of life in visually impaired baseball players: An Italian national survey." PLoS one 14, no. 6 (2019): e0218124.

Downloads

Published

2021-03-30

How to Cite

Migkotzidis, P., Kalaganis, F., Georgiadis, K., Chatzilari, E., Pehlivanides, G., Tsafaras, S., … Kompatsiaris, I. (2020). e-Vision: An AI-powered system for promoting the autonomy of visually impaired. European Journal of Creative Practices in Cities and Landscapes, 3(2), 33–53. https://doi.org/10.6092/issn.2612-0496/10707

Issue

Vol. 3 No. 2 (2020): Design for People’s Autonomy. Rights to Independent Living

Section

Main Section

License

Copyright (c) 2020 Panagiotis Migkotzidis, Fotis Kalaganis, Kostas Georgiadis, Elisavet Chatzilari, George Pehlivanides, Spyros Tsafaras, Kostas Monastiridis, Spiros Nikolopoulos, Ioannis Kompatsiaris

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.