Adapted leisure, information and remote assistance services via a television set, with advanced natural language voice communication functionalities for people with sensory disabilities and the aged.
A consortium coordinated by TMT Factory, and also comprising the Universitat Ramon Llull (Enginyeria i Arquitectura La Salle, UserLab), the Universidad Politécnica de Madrid (Artificial Intelligence Department) and the Universidad Carlos III de Madrid (IT Department), is undertaking an industrial research project geared to the planned investigation related to the e-inclusion and e-assistance strategic action lines of the Spanish National Programme for Information Society Service Technology, the aim of which is to acquire and apply new knowledge that could prove to be useful in terms of contributing to considerably improving the IntegraTV service.
The name of the project, IntegraTV-4all, is intended to reflect the work being carried out to obtain a new kind of television for everybody and to promote access to new technologies for people with handicaps, with suitable technological development geared to the possibilities open to them and their requirements, and the fostering of a design for all in the arena in question.
The Ministry of Industry, Tourism and Commerce is supporting the project through a substantial PROFIT grant, while the ONCE Foundation and the companies ATLAS and Fundosa Teleservicios are also cooperating therewith.
The IntegraTV-4all project aims to take IntegraTV in new directions, through the development and integration of a new module that could help to facilitate personal independence and the social integration of groups such as, primarily, those with sensory disabilities (blindness, visual impairments, deafness, hearing loss or speech impediments). However, the project’s result may also prove to be useful for people with physical or mental handicaps, as well as for the aged.
The pilot stage of the project encompasses three hotels from the Confortel chain (hotels especially equipped for the handicapped) and four home users, thus demonstrating the possibility of establishing a teleconference between users of the IntegraTV-4all service in non-homogeneous environments, along with the robust nature of the application.
Configuration for different sensory disabilities
Menu of services and content adapted to cater for visual defects (ASR + TTS)
Alerts for the deaf or those suffering from loss of hearing
Virtual presenter for the elderly (3D + ASR + TTS)
Stage 1 of the development of IntegraTV-4all includes a study of the design of the graphic interface and of the content. The IntegraTV-4all development team is in no doubt that, in addition to providing a user interface that is completely accessible to those with disabilities, it is vital to provide content adapted to such people. In that respect, research is being undertaken to determine what content is accessible to people with visual and/or hearing impairments. The content that has been found basically consists of children’s games and films for people with visual handicaps.
Stage 2 of the project involves the implementation of a basic interactive television service, making it possible to navigate through the menus using voice instructions. This basic IntegraTV-4all service uses the Verbio speech recognition and text to speech conversion software (made by the company ATLAS). The natural language speech recognition system allows for the menus available in the context applicable at any given time to be selected using voice instructions, along with a series of menus that are directly accessible from any point. The text to speech conversion software makes it possible to automatically synthesise the texts required for the system.
Stage 3 of the project entails the implementation of fundamental components of the advanced system, namely the free natural language dialogue system, the virtual announcer (developed by the Universitat Ramon Llull) and the virtual presenter. Consideration is being given to the incorporation of advanced human-machine interaction techniques in order to make it possible for certain IntegraTV-4all services (the alarm clock and the snooze function, for example) to be offered via an unstructured dialogue between the user and the system (see diagram).
To that end, it is necessary to:
1)
Develop the system’s ability to understand language;
2)
Facilitate the possibility of following proactive and joint-initiative dialogues (to which both the system and the user may contribute).
Examples
In order to make it easier to understand the way the project works, here are two examples of cases that could arise in relation to the pilot test in hotels:
The starting point for all the above was the results obtained by teams from the Universidad Politécnica de Madrid and the Universidad Carlos III de Madrid in the European projects ADVICE (virtual sales assistant for the complete customer service process in digital markets) and VIP-ADVISOR (virtual independent advisor for personal insurance and finance risk management). The resulting service functions as outlined in the diagram to which a link is provided below. The user communicates by means of an expression in natural language, which is analysed by a speech recognition mechanism. That mechanism’s output is processed in order to extract both its semantics and pragmatics, i.e. what the user says and their intention or purpose in doing so. That information is interpreted in the context of a conversation by an interaction manager, which decides how the system should respond, to which end it uses information related to the requested service and knowledge of the user’s characteristics or preferences. However, the use of user profiles is complicated and may be very limited in initial prototypes. Once a response has been decided upon, a natural language expression is generated and finally transmitted to the user by means of a voice synthesiser.
Product type: interactive television system Commission: analysis and development (including design and usability study) Architecture: .NET Communications: SOAP, natural language, ASR, TTS Dialogue management: threading model, software agents Development: Visual Studio C# Customer: Integra Interactive Website: www.integra.tv
Menu for handicapped users:
Virtual presenter:
Virtual presenter:
This project reached the finals of the IGC 2005 Digital Innovation Awards, promoted by the Internet Global Congress (www.igcweb.net). The IGC 2005 event took place in Barcelona between 6 and 10 June, with the aim of promoting innovation and knowledge in the digital society.
;)
;)
;)
;)
)
;){kind=link}
;){kind=link}
;){kind=link}
;){kind=link}