A subtitled world: uncovering tickertape synesthesia

The anthropologist Francis Galton, Charles Darwin’s cousin, was interested in the most notable faculties of the human mind. In 1883, he observed that some people visualized the speech of their interlocutor in their internal mental space. “Some few people see mentally in print every word that is uttered […], and they read them off usually as from a long imaginary strip of paper, such as is unwound from telegraphic instruments“. The ticker tape no longer exists today, but this particularity – called tickertape synesthesia (TTS), remains.

Synesthesia is a neurological phenomenon in which different senses are stimulated simultaneously while processing sensory information. It allows some artists to hear colors, see sounds, or even taste music. In the case of TTS, synesthetes can read someone’s voice.

Indeed, even though this characteristic was described more than a century ago, it is still poorly understood. A previous study estimated that up to 1.4% of the population could experience involuntary subtitles when hearing a human voice, but this figure remains uncertain. It’s challenging to detect TTS subjects, who are generally unaware that they are distinguishable from the average person. “Several participants in the study were shocked to learn that not everyone has built-in subtitles,” laughs Laurent Cohen. We taught them through our recruitment ad. Like most forms of synesthesia, TTS is a subjective phenomenon. We don’t know how to measure it objectively, as we would evaluate visual acuity, for example. Nor is it associated with outstanding abilities or incapacitating cognitive impairments. This makes it an exciting but invisible quality.

A thousand transcripts

To better understand the subjective experience of tickertape synesthesia, Fabien Hauw, and Laurent Cohen asked the 26 participants, whose native language was French, to fill out a questionnaire. Indeed, many questions were left answered: can SST be triggered by a foreign language, new words, or noises (sneezing, meowing, motor humming)? Are the subtitles formatted in a specific way (size, color, capitalization, special characters) for each individual?

The researchers’ results indicate that for 73% of the participants, synesthesia appeared during the acquisition of reading in childhood. With almost half of them, this characteristic proved to be both an advantage and a nuisance; it helps them memorize words but disrupts their attention in crowded places where many conversations occur simultaneously. Indeed, for 70% of the participants, TTS is an automatic process that cannot be controlled.

Some synesthetes report that the appearance of the captions can be affected by the context of verbalization, especially when the speaker is emotional; the words may then change color or size, depending on the intensity of the emotion. “Letters may be blurred or shaking when I’m moved,” said one participant.

Even more surprising: some subjects report that when they watch a foreign movie, a second level of subtitles – a product of their synesthesia – appears above the subtitles embedded in the video. Others have subtitled dreams and nightmares, which provide their oneiric activity with a cinematic dimension. Finally, since one-third of the participants knew of other cases of TTS in their family, the emergence of this form of synesthesia might have a genetic basis.

Powerful super-readers?

In literate people, specific mental processes make it possible to interpret words, sounds, and letters and give them meaning. To be effective, these processes are fine-tuned under genetic and environmental constraints during development. Thus, we can observe a wide variety in reading performance across individuals, ranging from dyslexia…to synesthesia. In that sense, researchers believe that TTS results from a very atypical development of literacy.

The researchers’ observations must be reproduced with a more extensive and diverse sample of subjects. “Thanks to this study, we can map the spectrum of perceptions that exist in TTS. Now, we want to ensure that it is related to an overdeveloped access to orthographic representations“, says Laurent Cohen. We will never know if synesthetic scribes in ancient Egypt subtitled their interlocutors in hieroglyphics. But by illuminating the mechanisms of reading, we may be able to help children for whom this acquisition remains challenging.