2 post-doctoral positions available: Translation of French Cued Speech.

Global framework
This work is a part of a French project which goal is the development of a phone terminal for hard of hearing people communicating with the specific Cued Speech language.

Cued Speech was developed by Dr. Cornett in 1967. Its purpose is to bring the natural oral language accessible to hearing impaired, by the intensive use of lip-reading. But lip-reading is ambiguous: for example, /p/ and /b/ are different phonemes with identical lip shape. Cornett proposed (1) to replace invisible articulators (such as vocal cords) that participate to the production of the sound by hand gestures, and (2) to keep the visible articulators (such as lips). Basically, it means complementing the lip-reading by various manual gestures, so that phonemes which have similar lip shapes can be differentiated. Thanks to the combination of both lip-shapes and manual gestures, each phoneme has a specific visual aspect. Such a “hand & lip-reading” becomes as meaningful as the oral message. The interest of CS is to use a code which is similar to oral language. As a consequence, it prevents hearing impaired people to have an under-specified representation of oral language and help them to learn to verbalize properly.

The CSs message is formatted into a list of consonant-vowel syllables (CV syllables). Each CV syllable is coded with a specific manual gesture and combined to the corresponding lip shape, so that the whole looks unique.

Post-doc proposals
Around the TELMA’s project, the Gispa-lab offers two post-doc prositions in order to work on :

• Image processing and data fusion: the candidate will have to test some of the algorithms that have been developed at the lab for lip contours extraction and for speech parameters estimation. A study about the estimated parameters for the purpose of speech data fusion models will have to be improved or redefined.
• Speech data fusion: the aim is to develop models for the fusion of the hand information and the lip information in order to translate sentences. This work could be based on the models already developped for syllbles and words recognition. The proposed model will have to take into account the non synchronization betwee the hand and the lip flows. Speech models coming from the automatic speech recgnition domain could be considered in order to improve the translation rate. A strong coordination and collaboration between both post-doctoral works will be necessary.

To apply , please send a resume by e-mail to: Alice Caplier caplier@enserg.fr (www.lis.inpg.fr) and/or Denis Beautemps Denis.Beautemps@gipsa-lab.inpg.fr