Auditory and Visual Information Facilitating Speech Integration PDF Download
Are you looking for read ebook online? Search for your book and save it on your Kindle device, PC, phones or tablets. Download Auditory and Visual Information Facilitating Speech Integration PDF full book. Access full book title Auditory and Visual Information Facilitating Speech Integration by Brandie Andrews. Download full books in PDF and EPUB format.
Author: Brandie Andrews Publisher: ISBN: Category : Languages : en Pages :
Book Description
Abstract: Speech perception is often thought to be a unimodal process (using one sense) when, in fact, it is a multimodal process that uses both auditory and visual inputs. In certain situations where the auditory signal has become compromised, the addition of visual cues can greatly improve a listener's ability to perceive speech (e.g., in a noisy environment or because of a hearing loss). Interestingly, there is evidence that visual cues are used even when the auditory signal is completely intelligible, as demonstrated in the McGurk Effect, in which simultaneous presentation of an auditory syllable "ba" with a visual syllable "ga" results in the perception of the sound "da," a fusion of the two inputs. Audiovisual speech perception ability varies widely across listeners; individuals integrate different amounts of auditory and visual information to understand speech. It is suggested that characteristics of the listener, characteristics of the auditory and visual inputs, and characteristics of the talker may all play a role in the variability of audiovisual integration. The present study explored the possibility that differences in talker characteristics (unique acoustic and visual characteristics of articulation) might be responsible for some of the variability in a listener's ability to perceive audiovisual speech. Ten listeners were presented with degraded auditory, visual, and audiovisual speech syllable stimuli produced by fourteen talkers. Results indicated substantial differences in intelligibility across talkers under the auditory-only condition, but little variability in visual-only intelligibility. In addition, talkers produced widely varying amounts of audiovisual integration, but interestingly, the talkers producing the most audiovisual integration were not those with the highest auditory-only intelligibility.
Author: Brandie Andrews Publisher: ISBN: Category : Languages : en Pages :
Book Description
Abstract: Speech perception is often thought to be a unimodal process (using one sense) when, in fact, it is a multimodal process that uses both auditory and visual inputs. In certain situations where the auditory signal has become compromised, the addition of visual cues can greatly improve a listener's ability to perceive speech (e.g., in a noisy environment or because of a hearing loss). Interestingly, there is evidence that visual cues are used even when the auditory signal is completely intelligible, as demonstrated in the McGurk Effect, in which simultaneous presentation of an auditory syllable "ba" with a visual syllable "ga" results in the perception of the sound "da," a fusion of the two inputs. Audiovisual speech perception ability varies widely across listeners; individuals integrate different amounts of auditory and visual information to understand speech. It is suggested that characteristics of the listener, characteristics of the auditory and visual inputs, and characteristics of the talker may all play a role in the variability of audiovisual integration. The present study explored the possibility that differences in talker characteristics (unique acoustic and visual characteristics of articulation) might be responsible for some of the variability in a listener's ability to perceive audiovisual speech. Ten listeners were presented with degraded auditory, visual, and audiovisual speech syllable stimuli produced by fourteen talkers. Results indicated substantial differences in intelligibility across talkers under the auditory-only condition, but little variability in visual-only intelligibility. In addition, talkers produced widely varying amounts of audiovisual integration, but interestingly, the talkers producing the most audiovisual integration were not those with the highest auditory-only intelligibility.
Author: Pamela Ver Hulst Publisher: ISBN: Category : Languages : en Pages :
Book Description
Abstract: Speech perception is often described as a unimodal process, when in reality it involves the integration of multiple sensory modalities, specifically, vision and hearing. Individuals use visual information to fill in missing pieces of auditory information when hearing has been compromised, such as with a hearing loss. However, individuals use visual cues even when auditory cues are perfect, and cannot ignore the integration that occurs between auditory and visual inputs when listening to speech. It is well known that individuals differ in their ability to integrate auditory and visual speech information, and likewise that some individuals produce clearer speech signals than others, either auditorily or visually. Clark (2005) found that some talkers in a study of the McGurk effect, produced much stronger 'integration effects' than did other talkers. One possible underlying mechanism of auditory + visual integration is the substantial redundancy found in the auditory speech signal. But how much redundancy is necessary for effective integration? And what auditory and visual characteristics make a good integration talker? The present study examined these questions by comparing the auditory intelligibility, visual intelligibility, and the degree of integration for speech sounds that were highly reduced in auditory redundancy, produced by 7 different talkers. Performance of participants under four conditions: 1) degraded auditory only, 2) visual only, 3) degraded auditory + visual, and 4) non-degraded auditory + visual, was examined. Results indicate across-talker differences in auditory and auditory + visual intelligibility. Degrading the auditory stimulus did not affect the overall amount of McGurk-type integration, but did influence the type of McGurk integration observed.
Author: Corinne D. Anderson Publisher: ISBN: Category : Languages : en Pages :
Book Description
Abstract: When people think about understanding speech, they primarily think about perceiving speech auditorily (via hearing); however, there are actually two key components to speech perception: auditory and visual. Speech perception is a multimodal process; i.e., combining more than one sense, involving the integration of auditory information and visual cues. Visual cues can supplement missing auditory information; for example, when auditory information is compromised, such as in noisy environments, seeing a talker's face can help a listener understand speech. Interestingly, auditory and visual integration occurs all of the time, even when the auditory and visual signals are perfectly intelligible. The role that visual cues play in speech perception is evidenced in a phenomenon known as the McGurk effect, which demonstrates how auditory and visual cues are integrated (McGurk and MacDonald, 1976). Previous studies of audiovisual speech perception suggest that there are several factors affecting auditory and visual integration. One factor is characteristics of the auditory and visual signals; i.e., how much information is necessary in each signal for listeners to optimally integrate auditory and visual cues. A second factor is the auditory and visual characteristics of individual talkers; e.g., visible cues such as mouth opening or acoustic cues such as speech clarity, that might facilitate integration. A third factor is characteristics of the individual listener; such as central auditory or visual abilities, that might facilitate greater or lesser degrees of integration (Grant and Seitz, 1998). The present study focused on the second factor, looking at both auditory and visual talker characteristics and their effect on auditory and visual integration of listeners. Preliminary results of this study show considerable variability across talkers in the auditory only condition, suggesting that different talkers have different degrees of auditory intelligibility. Interestingly, there were also substantial differences in the amount of audiovisual integration produced by different talkers that were not highly correlated with auditory intelligibility, suggesting talkers who have optimal auditory intelligibility are not the same talkers that facilitate optimal audiovisual integration.
Author: Kaisa Tiippana Publisher: Frontiers Media SA ISBN: 2889195481 Category : Psychology Languages : en Pages : 265
Book Description
Speech is multisensory since it is perceived through several senses. Audition is the most important one as speech is mostly heard. The role of vision has long been acknowledged since many articulatory gestures can be seen on the talker's face. Sometimes speech can even be felt by touching the face. The best-known multisensory illusion is the McGurk effect, where incongruent visual articulation changes the auditory percept. The interest in the McGurk effect arises from a major general question in multisensory research: How is information from different senses combined? Despite decades of research, a conclusive explanation for the illusion remains elusive. This is a good demonstration of the challenges in the study of multisensory integration. Speech is special in many ways. It is the main means of human communication, and a manifestation of a unique language system. It is a signal with which all humans have a lot of experience. We are exposed to it from birth, and learn it through development in face-to-face contact with others. It is a signal that we can both perceive and produce. The role of the motor system in speech perception has been debated for a long time. Despite very active current research, it is still unclear to which extent, and in which role, the motor system is involved in speech perception. Recent evidence shows that brain areas involved in speech production are activated during listening to speech and watching a talker's articulatory gestures. Speaking involves coordination of articulatory movements and monitoring their auditory and somatosensory consequences. How do auditory, visual, somatosensory, and motor brain areas interact during speech perception? How do these sensorimotor interactions contribute to speech perception? It is surprising that despite a vast amount of research, the secrets of speech perception have not yet been solved. The multisensory and sensorimotor approaches provide new opportunities in solving them. Contributions to the research topic are encouraged for a wide spectrum of research on speech perception in multisensory and sensorimotor contexts, including novel experimental findings ranging from psychophysics to brain imaging, theories and models, reviews and opinions.
Author: Douglas Burnham Publisher: Psychology Press ISBN: 1135471959 Category : Psychology Languages : en Pages : 338
Book Description
This volume outlines some of the developments in practical and theoretical research into speechreading lipreading that have taken place since the publication of the original "Hearing by Eye". It comprises 15 chapters by international researchers in psychology, psycholinguistics, experimental and clinical speech science, and computer engineering. It answers theoretical questions what are the mechanisms by which heard and seen speech combine? and practical ones what makes a good speechreader? Can machines be programmed to recognize seen and seen-and-heard speech?. The book is written in a non-technical way and starts to articulate a behaviourally-based but cross-disciplinary programme of research in understanding how natural language can be delivered by different modalities.
Author: Nicholas Altieri Publisher: Universal-Publishers ISBN: 1599423618 Category : Languages : en Pages :
Book Description
Auditory and visual speech recognition unfolds in real time and occurs effortlessly for normal hearing listeners. However, model theoretic descriptions of the systems level cognitive processes responsible for integrating auditory and visual speech information are currently lacking, primarily because they rely too heavily on accuracy rather than reaction time predictions. Speech and language researchers have argued about whether audiovisual integration occurs in a parallel or in coactive fashion, and also the extent to which audiovisual occurs in an efficient manner. The Double Factorial Paradigm introduced in Section 1 is an experimental paradigm that is equipped to address dynamical processing issues related to architecture (parallel vs. coactive processing) as well as efficiency (capacity). Experiment 1 employed a simple word discrimination task to assess both architecture and capacity in high accuracy settings. Experiments 2 and 3 assessed these same issues using auditory and visual distractors in Divided Attention and Focused Attention tasks respectively. Experiment 4 investigated audiovisual integration efficiency across different auditory signal-to-noise ratios. The results can be summarized as follows: Integration typically occurs in parallel with an efficient stopping rule, integration occurs automatically in both focused and divided attention versions of the task, and audiovisual integration is only efficient (in the time domain) when the clarity of the auditory signal is relatively poor--although considerable individual differences were observed. In Section 3, these results were captured within the milieu of parallel linear dynamic processing models with cross channel interactions. Finally, in Section 4, I discussed broader implications for this research, including applications for clinical research and neural-biological models of audiovisual convergence.
Author: Pascal Belin Publisher: Springer Science & Business Media ISBN: 1461435854 Category : Medical Languages : en Pages : 384
Book Description
This book follows a successful symposium organized in June 2009 at the Human Brain Mapping conference. The topic is at the crossroads of two domains of increasing importance and appeal in the neuroimaging/neuroscience community: multi-modal integration, and social neuroscience. Most of our social interactions involve combining information from both the face and voice of other persons: speech information, but also crucial nonverbal information on the person’s identity and affective state. The cerebral bases of the multimodal integration of speech have been intensively investigated; by contrast only few studies have focused on nonverbal aspects of face-voice integration. This work highlights recent advances in investigations of the behavioral and cerebral bases of face-voice multimodal integration in the context of person perception, focusing on the integration of affective and identity information. Several research domains are brought together. Behavioral and neuroimaging work in normal adult humans included are presented alongside evidence from other domains to provide complementary perspectives: studies in human children for a developmental perspective, studies in non-human primates for an evolutionary perspective, and studies in human clinical populations for a clinical perspective. Several research domains are brought together. Behavioral and neuroimaging work in normal adult humans included are presented alongside evidence from other domains to provide complementary perspectives: studies in human children for a developmental perspective, studies in non-human primates for an evolutionary perspective, and studies in human clinical populations for a clinical perspective. Several research domains are brought together. Behavioral and neuroimaging work in normal adult humans included are presented alongside evidence from other domains to provide complementary perspectives: studies in human children for a developmental perspective, studies in non-human primates for an evolutionary perspective, and studies in human clinical populations for a clinical perspective. Several research domains are brought together. Behavioral and neuroimaging work in normal adult humans included are presented alongside evidence from other domains to provide complementary perspectives: studies in human children for a developmental perspective, studies in non-human primates for an evolutionary perspective, and studies in human clinical populations for a clinical perspective.
Author: Gérard Bailly Publisher: Cambridge University Press ISBN: 1107006821 Category : Computers Languages : en Pages : 507
Book Description
This book presents a complete overview of all aspects of audiovisual speech including perception, production, brain processing and technology.
Author: Patricia J. Brooks Publisher: SAGE Publications ISBN: 1483389774 Category : Language Arts & Disciplines Languages : en Pages : 1471
Book Description
The progression from newborn to sophisticated language user in just a few short years is often described as wonderful and miraculous. What are the biological, cognitive, and social underpinnings of this miracle? What major language development milestones occur in infancy? What methodologies do researchers employ in studying this progression? Why do some become adept at multiple languages while others face a lifelong struggle with just one? What accounts for declines in language proficiency, and how might such declines be moderated? Despite an abundance of textbooks, specialized monographs, and a couple of academic handbooks, there has been no encyclopedic reference work in this area--until now. The Encyclopedia of Language Development covers the breadth of theory and research on language development from birth through adulthood, as well as their practical application. Features: This affordable A-to-Z reference includes 200 articles that address such topic areas as theories and research tradition; biological perspectives; cognitive perspectives; family, peer, and social influences; bilingualism; special populations and disorders; and more. All articles (signed and authored by key figures in the field) conclude with cross reference links and suggestions for further reading. Appendices include a Resource Guide with annotated lists of classic books and articles, journals, associations, and web sites; a Glossary of specialized terms; and a Chronology offering an overview and history of the field. A thematic Reader’s Guide groups related articles by broad topic areas as one handy search feature on the e-Reference platform, which includes a comprehensive index of search terms. Available in both print and electronic formats, Encyclopedia of Language Development is a must-have reference for researchers and is ideal for library reference or circulating collections. Key Themes: Categories Effects of language on cognitive development Fundamentals, theories and models of language development Impairments of language development Language development in special populations Literacy and language development Mechanisms of language development Methods in language development research Prelinguistic communicative development Social effects in language acquisition Specific aspects of language development
Author: Brandie Andrews
Author: Brandie Andrews
Author: Pamela Ver Hulst
Author: Corinne D. Anderson
Author: Kaisa Tiippana
Author: Douglas Burnham
Author: Ruth Campbell
Author: Nicholas Altieri
Author: Pascal Belin
Author: Gérard Bailly
Author: Patricia J. Brooks