Audio feature: Oh, I See

horse_hms

Seeing with your ears.

An audio feature I produced over the summer for Pod Academy, exploring the development of the vOICe technology and it’s impact on blind users. The vOICe is a computer program developed by dutch engineer Dr Peter Meijer which essentially converts images into sound. Through training and experience blind users can learn to interpret these sounds as a sort of ‘synthetic vision’. The piece explores the technology from the perspective of blind user Pat Fletcher, and uncovers some of the science and technology behind its use with it’s creator Dr Peter Meijer and cognitive psychologist Dr Michael Proulx (University of Bath).

It was my thought that technology and the computer would be my way out of blindness.

-Pat Fletcher, vOICe user

Download it HERE

Pat Fletcher
Blind user Pat Fletcher uses the vOICe with video glasses, stereo headphones and a portable notebook computer (Image: Barbara Schweizer )

Essentially, the software takes spatial information captured by a camera and converts this into a coded soundscape. Users can then learn how to decode this auditory signal into a visual one thanks to a process known as ‘sensory substitution’, where information from one sense is fed to the brain via another. Fundamentally what the vOICe is doing is re-routing information usually obtained by the eyes and delivering it through another sense organ, the ears.

Although the neuroscience and psychology behind the technology is still largely unknown, it is thought that the visual cortex is eventually recruited to process the incoming auditory information and through experience, is able to decode it as spatial / visual information. There’s a great article over at New Scientist that goes into greater depth about the neuroscience behind it  – including a useful diagram depicting how the technology works.

The software is currently freely available and can be used with virtually any imaging device, from webcams to camera-mounted glasses – there’s even an android version available for mobile devices! With the increasing prevalence of mobile computing, the vOICe technology is liberating users from their blindness, allowing them to step outside and experience the world through a completely new visual perspective.

For more information visit: http://www.seeingwithsound.com/ where you can experiment with the vOICe for youself and learn more about how it works. I’ve also prepared a page with a collection of images as heard through the vOICe software, including some featured within the piece above.

Music

  1. Hypermagic – Start Again Start
  2. Ed Prosser – Untitled
  3. – – b31
  4. No Color – L’Aube
  5. Hpermagic – Pico Bisco
  6. Ed Prosser – Untitled
  7. Marcel Pequel – Four

Freesound Credits (freesoundarchive.com)

  1. Alarm Clock – 14262__xyzr-kx__alarm-clock
  2. Camera Shutter – 16071__heigh-hoo__nikonf4
  3. Data sound – 3647__suonho__futuretrocomputing-10-suonho

Video: The Alkali Metal Match

TFTPR-Alkali Metal Match_03

Lighting a match with a single drop of water, sort of…

A break from audio related work – here’s a little video I made recently for the Ri Channel. It’s not exactly the most of significant of experiments, but when someone commented on a previous YouTube video asking whether it would be possible to ignite a match with a piece of Sodium and a single drop of water, we thought we should have a go.

The video was shot in a morning on a single Panasonic AG-HMC151, in the Royal Institution’s legendary prep room with demo Technician Andy Marmery.