Is Seeing Perceiving? Do Visual Cues Enhance Auditory Attention and Comprehension in a Mixed Speech Background?

Overview

Objectives/Goals The objective of this study is to test if visual cues such as images will improve the comprehension of information when two sentences are spoken at once. Methods/Materials To test my hypotheses, I conducted 2 sets of hearing tests, each consisting of a video recording of 2 spoken sentences combined with an image. Test #1 contains 2 sentences (sentences 1 and 2) that are dissimilar in words and sentence structure. Test #2 contains 2 different sentences (sentences 3 and 4) that are similar in their properties. Among the 28 male and 32 female subjects I recruited, I randomly divided them into 3 test groups, namely groups A, B, and C. Group A was shown images that were relevant to sentences 1 and 3. The images preceded the recorded sentences in the video, made using iMovie; while group B was shown images that were relevant to sentences 2 and 4. Group C was shown blank images that preceded the same recordings as used for groups A and B. In other words, all 3 groups were subjected to the same voice recordings, but to different visual cues prior to the mixed conversations. Results 1). In general, the performance between the multiple-choice test and written test is comparable. 2). In mixed sentences 1 and 2, the image of a dancer shown to group B significantly improved the understanding of sentence 2. The visual cue of a brown fox also improved the understanding of sentence 1 for subjects in group A, although to a lesser extent. The reason for this difference is unclear, possibly due to the dominant effect of dancer image and the content of sentence 2 on my subjects. 3). In hearing test #2, both visual cues (A: tiger; B: zebra) helped the understanding of sentences 3 and 4. 4). The hearing score of group C did not favor either sentence in both hearing tests. Group C scored lowest overall, due to the fact that they did not have any visual cues helping them. My data suggest that visual cues help enhance the understanding and processing in a mixed speech background, particularly when sentences share similar sentence structure. 5). There is no overall gender difference in the auditory comprehension performance. Conclusions/Discussion Visual cues improve auditory attention and perception in a mixed speech background under most circumstances, particularly when the competing sentences are very similar. There is no overall gender difference in auditory perception of mixed sentences.

Awards (1)

• Third Place

Competition history

• CSEF 2016 Behavioral & Social Sciences · Entry S0422

Resources

• CSEF project summary ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Behavioural and Social Sciences projects
• CSEF Behavioural and Social Sciences projects
• Every CSEF 2016 project
• CSEF Third Place winners