Late last week I spent some energy questioning and answering my current understanding of dual-coding theory and it’s distinction between verbal and non-verbal, vs. visual and non-visual.
Dual-coding theory suggests that learning can be actually be enhanced when cognitive load is increased in a specific way, such that both auditory and visual coding is activated simultaneous, and in concert. This is only true, dual-coding warns, when the information does not compete. This is where the distinction between verbal and non-verbal becomes important. Two forms of verbal information–visual verbal and auditory verbal–is likely to compete, even if one is spoken and one is heard. Another example of competing information would be visual non-verbal and visual verbal information, but not visual non-verbal and auditory verbal. Though I haven’t seen any studies of it, I daresay because we can easily discern lyrics from a song, non-verbal auditory and verbal auditory do not compete.
As I tried to understand why certain combinations of information compete and others don’t–and indeed are complementary–I turned to a book I’d read earlier this year, Stanislas Dehane’s “Reading in the Brain”. Dehane begins by explains that reading takes advantage of massive parallel processing (I love the daemons/pandemonium metaphor). He then explains the dual-route theory, which postulates that reading proceeds predominantly along one of two routes: phonological (or graphophonological) (visual–>auditory–>lexical) or orthographic (or lexicosemantic) (visual –> lexical), depending on the word being read:
All writing systems oscillate between an accurate representation of sound and the fast transmission of meaning. … Two information processing pathways coexist and supplement each other while we read. When words are very regular, rare, or novel we preferentially process them using a “phonological route”, in which we decipher the letter string, then convert it into pronunciation, and finally attempt to access the meaning of the sound patter (if any). Conversely, when we are confronted with words that are frequent or whose pronunciation is exceptional, our reading takes a direct route that first recovers the identity and meaning of the word and then uses the lexical information to recover it’s pronunciation.
(Dehane 38-39)
Research beginning and the late 80s and proceeding to today have supported this theory. Historic brain imaging experiments in 1989 from Petersen et al showed that different regions of the brain were used when listening to words (superior temporal region) vs. silently reading words (left occipito-temporal region, aka the brain’s letterbox–different from areas of brain which code broader visual stimuli). Some silent reading tasks then transmit to the superior temporal region for coding. (67-74)
Initial input pathway differs depending on whether we hear or read a word, but cortical activitiy later converges onto the same set of language areas.” And appears to extend out (and back, and out again) from there, apparently activating more areas of the brain than seems “necessary” to understand, well beyond the actual recognition of the word. (103-104)
That doesn’t mean we only use auditory areas when listening to words; we also may use them as the phonological route suggested by dual-route theory. This is described by later fMRI-based brain imaging experiments that found…
“Common words, as well as words whose spelling is irregular, head straight for the lexical areas of the middle temporal lobe… Other words, whether they are regular, rare, or unknown, are first pronounced mentally using auditory areas of the superior temporal lobe before a meaning is attached to them.” (116)
In terms of speed, it looks like the direct lexical route is faster (words we know; ~200 milliseconds) that the auditory first route (words we have to sound out; ~400 milliseconds).
Because either may be used, and because the route is largely based on the reader’s prior knowledge, it seems important not to interfere–not even risk interference with whichever route is taken. This supports dual-coding’s rule against competing information (verbal and verbal). It doesn’t complete the picture, however, or, if it does I haven’t quite discerned the layout. One (of many) questions I have: what is the impact on activation time of these two routes if the task is burdened with auditory stimuli, such as narration (either conflicting or coinciding)? I would guess that in instances where the the word is first “pronounced mentally” would be more subject to interference or delay from auditory stimuli or narration.
