The aim of this project is to study the real-time transmission
between two communication modes, speech and writing in human interaction
with a method called "print interpreting". It means translation of spoken
language and accompanying significant audible information into written
text simultaneously with the talk. The text is typed on a computer and
displayed on a screen where the letter-by-letter emerging text is visible.
Print interpreting is needed as a communication aid for people with hearing
disability to give them access to the speech. Since they have acquired the
language in a hearing speech culture and usually can speak it, they need an
interpretation which is as close as possible to the original speech. This
interpretation must also give an impression of the speaker and the linguistic
variation. The challenges of print interpreting are the demands of simultaneity
(requiring a high production rate) and verbatim transcription. Another
important challenge is to transfer all the relevant auditory information
(including non-language sounds from surroundings, etc.) into a visible modality
which is understandable to the hearing impaired.
The objectives of the study are 1) to investigate the process of print
interpreting and 2) the comprehensibility of the interpretation; and 3) to develop
new technology and methods for analyzing and supporting print interpreting. The
process means in the narrow sense the real-time conversion act and the changes in
the message; in the broader sense it covers the whole communicative event, including
the activity of interpreting and the actions of the participants and their interaction.
The comprehensibility will be examined in terms of readability and coherence. The
main research methods are textual and multimodal analysis, and eye movement analysis.
Because the research problems are multidisciplinary, they will be studied in an
interdisciplinary collaboration combining approaches from Linguistics, Translation
Studies (especially Interpreting Studies), and Computer Sciences.
The practical aim of the study is to develop new technological solutions and to
improve the accessibility of communication. Results on the reading process can provide
valuable information to develop better ways of rendering the text, and thereby help in
making the hearing impaired persons more equal partners in the ubiquitous communication
situation. In addition, the study will contribute to deeper understanding of the
relationship between writing and speaking, verbal and non-verbal communication, and
produce new information of their interchangeability in various media.