Creative Synaesthesia – If You See What I’m Saying

I discovered the Reactable, a new music-making interface, via City of Sound and Peter Marsh.

Why do I find this so fascinating? I think it’s the way it opens up new creative possiblities via artificial synaesthesia.

According to the scientists, ‘true’ synaesthesia is a neurological condition in which one sense is involuntarily translated into another – e.g. colours are experienced as sounds or vice versa. It is popularly associated with psychedelic drugs, but can also result from a stroke, blindness or deafness. I encountered synaesthesia in my work as a hypnotherapist, as it’s a fairly common occurrence in trance subjects.

Synaesthesia and Creativity

Less extreme versions of synaesthesia, sometimes called ‘pseudo-synaesthesia’, are reported by many people as part of their normal thinking processes. This kind of everyday synaesthesia seems to be particularly common among artists and other creative types. Like a lot of poets, I experience a kind of grapheme-colour synaesthesia, whereby words (and numbers) are associated with particular colours. Louis MacNeice describes the phenomenon in his poem ‘When we were children’:

When we were children words were coloured
(Harlot and murder were dark purple)
And language was a prism, the light
A coloured inlay on the grass,

Another of my favourite examples of synaesthesia is the artist and writer Mervyn Peake. A brilliant draughtsman and illustrator, while writing his novel Titus Groan Peake made sketches of characters in the margin:

As I went along I made drawings from time to time which helped me to visualise the characters and to imagine what sort of things they would say. The drawings were never exactly as I imagined the people, but were near enough for me to know when their voices lost touch with their heads.
(Mervyn Peake, ‘How a Romantic Novel was Evolved’)

Here the sketches are not merely decorative, but are integral to the process of generating dialogue – synaesthetic ‘talking heads’. For more good examples of synaesthetic creativity have a look at Wikipedia’s list of famous synaesthetes.

Back to the video – personally, I’m a musical imbecile but even I could probably make music out of this contraption, since I’m a lot more confident at arranging images than sounds. The interface would create an artificial synaesthesia, allowing me to translate visual arrangements into auditory soundscapes. It helps that I know that this is what a lot of musicians do naturally. My brother Paul is such a skilled musician that last weekend he very nearly got away with playing The Beatles’ ‘Something’ on a ukelele. Ages ago, I remember asking him about the stories of Mozart claiming he could hear an entire piece of music ‘all at once’ – Paul said “Of course he could, you just have to visualise it”. I’ve heard the same thing from several musicians I’ve coached.

Composer Michael Colgrass on Creative Synesthesia

Here’s US composer Michael Colgrass being interviewed by NLP teacher Robert Dilts, about musical composition:

Colgrass: Now you start constructing… to a certain extent and building. You can actually sit back and start to see blocks coming together. Sometimes people say “How do you write pieces?” And I’ll say… “You build them.” You do write with a pencil… that’s the mark you make. But you do build, you construct…
And a certain detachment begins to take place too… Because as you detach yourself, you are getting a Gestalt view of what’s going on here, see. Because this piece is going to last twenty minutes but you’ve got to be able to see it… ‘Swooch,’ as finished. You’ve got to be able to see from here to here… You can’t sing through twenty minutes every time you want to check through something here at the seventeenth minute…

Dilts: You store the piece visually so you can see it all at once. If you stored it kinesthetically or auditorily you’d have to go through it sequentially. But if it’s processed visual to kinesthetic you can rapidly go through the whole complex of feelings. Do you automatically see the notes for each sound?

Colgrass: Well, actually these are amorphous images that I am speaking of now, not the eighth notes or sixteenth notes or b-flats… It’s kind of like a painting, but not exactly. It’s an abstract image.
(From Tools for Dreamers, by Dilts, Epstein and Dilts)

Colgrass goes on to describe how he uses this kind of synaesthesia to teach children the basics of musical composition. It’s interesting to see that he begins, not from sound or images, but with kinaesthetics – movement and sensation:

Colgrass: I may start warming them up by telling them to move around, change their posture and position and start making any old sounds. (I myself will often stand on my head when I’m preparing to compose.) And the room becomes cacophonous with noise from people howling and screeching and grunting, and clicking their mouths. And I ask them to think of a mark they could put on the blackboard that would represent that sound.

From the initial kinaesthetics (“move around, change their posture and position”) Colgrass elicits “any old sounds” which he then asks the children to translate into a visual image (a mark on the blackboard). He is very clear about the importance of these synaesthetic connections in the composition process:

Colgrass: By now they are seeing too. I think this is important because people often have a lot of trouble hearing. As you have pointed out, we’re more visual than auditory in North America. So, when they can see the sound, as it were, then they can hear the sound better. That’s why I go to the blackboard with it.

The Reactable and Synaesthetic Feedback

Looking at the video again, we can see that it facilitates a synaesthetic feedback loop similar to the one described by Colgrass:

1. When using the interface, you start with kinaesthetics, manipulating the blocks that seem to float/slide on top of the screen
2. The movement of the blocks is simultaneously translated into pulsing sound and images, a great way of associating them in your mind/body
3. As with all music, as soon as you hear it, you get a feel (kinaesthetic) for whether you like it or not. If you like it, you’re likely to experience some head-nodding and foot-tapping (more kinaesthetics).
4. Which leads you to adjust the blocks
5. Which changes the sound/images
6. Which in turn leads you back round to feelings, head-noddings and foot-tappings.

Here’s a diagram to help you (ahem) see what I mean:

5 ways to cultivate creative synaesthesia:

1. Are you doing it already?
Notice whether synaesthesia is already part of your creative process:

• For instance, if you are drawing, notice how your body feels, whether your feet start tapping to an imaginary rhythm or tune.
• If you are writing, do you experience the words as if ‘listening in’ to an inner voice, or do they form images in your mind – or both?
• If you are singing, dancing or otherwise engaged in physical performance – do images or colours go through your awareness as you perform?

Often, just noticing these elements of your creative process can make them more pronounced and effective. You might even want to consciously use them as ‘cues’ to access or intensify a creative state of mind.

2. Album covers
Think of your favourite album – what comes to mind first? Chances are the album cover pops into your head as a visual ‘icon’ or shorthand for the whole album. I find it hard to think of Primal Scream’s Screamadelica without picturing that manic little multicoloured spider-sun jigging about, or Ziggy Stardust without seeing Bowie loitering around that phone box.

• Think of a current creative project, in any medium – music, writing, a piece of design, film, whatever. If you had to design an album cover for the project, what would it look like? What kind of images spring to mind? What kind of feeling do you get from them?

3. Ask yourself ‘What if?’

• Look at an image and ask yourself – “If this were a sound, what would it sound like?” or “If it were a feeling, what would it feel like?”.
• Listen to a piece of music and ask “If this were a picture, what would it look like?” or “If it were a person, how would s/he talk? What would s/he say?”
• Notice a feeling or sensation in your body – if you had to paint it, what colour/shape/size would it be? If you had to play it on an instrument, how would it sound?

Don’t strain or try to think of something clever – just notice what images, sounds or feelings naturally come to mind when you ask yourself the questions.

4. Use Mervyn Peake’s strategy (for writers)

• If you are writing a story, stop and sketch pictures of your characters – don’t worry if the drawings aren’t perfect, you don’t need to show them to anyone else. Look at the drawings, notice how you feel as you look at each face, and ask yourself “What kind of voice does that face have? What would that head say?”

5. Use Michael Colgrass’ strategy

• Start ‘warming up’ by moving around, limbering up, noticing feelings and sensations in your body, maybe noticing whether it wants to walk, dance, sway etc.
• What sounds or words come to mind as your body moves?
• How would those sounds look if you drew them? Draw them on a piece of paper, no matter how crudely.

If you start to feel silly or self-conscious doing this, remember Michael Colgrass’ words:

“Now, what you have done here is exactly what a composer does, no different. Except, you have not specified exactly how high or how low the sounds are. How long, how sharp, how thin, how loud, how soft. A composer has notations for those”

Over to you…

Do you have any experiences of creative synaesthesia you’d like to share? Or tips for using synaesthesia in creative work?