Are our experiences really real?

18.10.16

A recent BBC Horizon documentary entitled ‘Is Seeing Believing?’ looked at how our senses work to allow us to understand the world around us. This fascinating study revealed a number of very interesting observations about how human senses work and particularly how the brain interprets information.

We all tend to think that what we see, hear, touch, smell and taste are all an objective record of reality. However, this is not the case. The brain interprets the information it receives and effectively constructs a perception of reality inside the mind of each individual based on their previous experiences. A great example that many people will remember is the famous photo of ‘that dress’ that was circulated on social media in February 2015. This photo was the cause of many heated disagreements up and down the country as people argued over what colour the dress was. Some saw the dress and white and gold whereas others saw it as black and blue. This confusion was down to the specific lighting conditions in the photo and how people’s brains interpreted what they were seeing in different ways. It turns out the dress was black and blue (although I still see white and gold!)

90% of what we think we are seeing is our brain filling in the gaps

Numerous examples in the Horizon film showed how our perception of reality is largely constructed in our minds rather than objectively measured by our senses. It was stated that only around ten percent of the information processed by our visual cortex comes from the eyes. That means around ninety percent of what we think we are seeing is our brain filling in the gaps. It has been estimated that the quality of the human eye is only equivalent to a two megapixel camera. The smartphone in your pocket can see better than you. Our brains use our existing knowledge and experience to supplement and enhance what we actually see. This is why humans often see faces in inanimate objects. Our brains process the information from our eyes and, if it fits the pattern of a face, then our brain tells us we are looking at a face.

This very clever aspect of how our brains work can also be used to trick us. For centuries illusionists have used particular techniques to fool us into thinking we saw something that we did not. The Horizon film showed an example of an illusion where a magician throws a ball up in the air and catches it several times. On the fourth or fifth throw he only pretends to throw the ball but, to the viewer, it appears as if the ball leaves the magician’s hand and simply disappears into thin air. The initial real throws prime the brain of the viewer to expect the ball to be thrown into the air. When the pretend throw occurs, the viewer’s brain fills in the gap to make them think they saw the ball leave the magician’s hand. The viewer might swear blind they saw the ball but a slow motion replay reveals the ball never left his hand.

Kinneir Dufort interpreting coloured liquids

It is not just our visual senses that can be fooled. When conflicting information from multiple senses is combined, our brains can struggle to interpret what they are receiving. An example shown was a person given a selection of coloured drinks. However, the colours of the drinks did not match the flavours (eg. A strawberry flavoured drink was coloured green, a lime flavoured drink coloured blue etc.) The test subject was unable to identify the flavours of the drinks because the brain’s expectations had been misdirected by the colours.

What do these insights mean for those of us who design experiences?

Firstly, we must recognise that there is no objective reality. Users interpret and understand the experiences we create based on their individual pre-existing knowledge and experience. The importance of direct user engagement can never be underestimated. Of course we usually cannot design for specific individuals but we can identify common expectations within a given user group and use those as a basis.

Secondly, we must ensure that where information is received via multiple senses, such as sound and vision, that information is always consistent.

To make new experiences easily understood, we should try to use existing mental models and paradigms to ensure users make sense of the processes and interactions we present them with. But this does not mean we cannot create new models or experiences. The brain is very malleable and can easily be primed to a new expectation. So where we do produce an experience which is novel or different, we need to deliberately provide a means to prime the user’s expectations to how that experience behaves.

Kinneir Dufort The Worlds Magnetic Field

The second part of the Horizon film looked at how technology is being used to enhance existing senses or even to create new senses.

A research team in Germany have created a prototype device which uses haptic feedback to tell the user where magnetic North is in relation to their physical orientation. This is effectively creating a new sense; a built in compass. They demonstrated how using this device a user was able to navigate tracks without looking. This study was based around understanding how well the brain can adapt to new sensory input and how long it takes for the brain to use that input intuitively rather than through conscious effort on the part of the user.

Another example used similar technology to address a very specific need. Military helicopter pilots can easily lose their sense of orientation. Unless they can see the horizon or are looking at the right instrument in the cockpit, it is easy to lose their bearings and in battlefield situations this can result in fatal accidents. The American military are developing a system of haptic feedback positioned at various points over the pilot’s body so he can ‘feel’ the helicopter’s orientation while his eyes are busy looking elsewhere. A demonstration in a simulator showed a blindfolded pilot performing a perfect landing using only this haptic system as sensory input.

The combined application of experience design and technology when paired with clear scientific understanding of how our senses and brain work in unison to construct our reality presents amazing opportunities. Using technology to redefine how we interact with the world around us in a way that seems intuitive and understandable allows us create new experiences, improve efficiency, improve safety and even save lives.

Phil Jenkins  - Head of UX

Phil will be in London at Interact London 2016 asking the question 'How will people interact with technology in the future and how do we design for it today?