Audio feature: Oh, I See

Seeing with your ears.

An audio feature I produced over the summer for Pod Academy, exploring the development of the vOICe technology and it’s impact on blind users. The vOICe is a computer program developed by dutch engineer Dr Peter Meijer which essentially converts images into sound. Through training and experience blind users can learn to interpret these sounds as a sort of ‘synthetic vision’. The piece explores the technology from the perspective of blind user Pat Fletcher, and uncovers some of the science and technology behind its use with it’s creator Dr Peter Meijer and cognitive psychologist Dr Michael Proulx (University of Bath).

It was my thought that technology and the computer would be my way out of blindness.

-Pat Fletcher, vOICe user

Download it HERE

Pat Fletcher
Blind user Pat Fletcher uses the vOICe with video glasses, stereo headphones and a portable notebook computer (Image: Barbara Schweizer )

Essentially, the software takes spatial information captured by a camera and converts this into a coded soundscape. Users can then learn how to decode this auditory signal into a visual one thanks to a process known as ‘sensory substitution’, where information from one sense is fed to the brain via another. Fundamentally what the vOICe is doing is re-routing information usually obtained by the eyes and delivering it through another sense organ, the ears.

Although the neuroscience and psychology behind the technology is still largely unknown, it is thought that the visual cortex is eventually recruited to process the incoming auditory information and through experience, is able to decode it as spatial / visual information. There’s a great article over at New Scientist that goes into greater depth about the neuroscience behind it  – including a useful diagram depicting how the technology works.

The software is currently freely available and can be used with virtually any imaging device, from webcams to camera-mounted glasses – there’s even an android version available for mobile devices! With the increasing prevalence of mobile computing, the vOICe technology is liberating users from their blindness, allowing them to step outside and experience the world through a completely new visual perspective.

For more information visit: http://www.seeingwithsound.com/ where you can experiment with the vOICe for youself and learn more about how it works. I’ve also prepared a page with a collection of images as heard through the vOICe software, including some featured within the piece above.

Music

  1. Hypermagic – Start Again Start
  2. Ed Prosser – Untitled
  3. – – b31
  4. No Color – L’Aube
  5. Hpermagic – Pico Bisco
  6. Ed Prosser – Untitled
  7. Marcel Pequel – Four

Freesound Credits (freesoundarchive.com)

  1. Alarm Clock – 14262__xyzr-kx__alarm-clock
  2. Camera Shutter – 16071__heigh-hoo__nikonf4
  3. Data sound – 3647__suonho__futuretrocomputing-10-suonho
Advertisements

Pod Academy – In The Morgue

I’ve recently started producing podcasts for a new non-profit organisation called Pod Academy – they release weekly podcasts on academia and research, covering everything from the arts and culture to science and the environment. There’s a really nice range of subjects covered by the podcasts and their library is growing on a weekly basis. You can browse what they have to offer here.

Auto-erotic Asphyxiation

My latest offering takes a trip around the morgue of the Sunderland Royal Infirmary, with pathologist Dr Stuart Hamilton as a guide. The piece provides a glimpse into mortuary life,  from working with the dead on a daily basis, to dealing with cross dressing ‘auto-erotic asphyxiation’ fatalities.

You can listen to the piece over at the Pod Academy website here. Or download it here. The piece takes material recorded for my larger documentary piece ‘The D-Word’.

If you want to stay up to date with the podcasts you can subscribe to their podcast feed via iTunes.

Chris Watson on Noise

Interview

Last year I interviewed sound recordist Chris Watson on the subject of noise for a piece exploring the use of birdsong at Alder Hey Children’s Hospital in Liverpool.

Chris is one of the worlds leading sound recordists and is well known for his work with the BBC Natural History Unit, including the recent Frozen Planet series.

There was a lot of interesting discussion during this interview about the nature of noise pollution and the considerable threat it poses to our quality of life. Worrying still, it appears that our noisy modern world is drowning out the natural soundscape and interfering with species of wildlife that rely on sound for communication.

What seems to be most alarming is that we’re largely ignoring this problem – our world certainly isn’t getting any quieter – and with more of us living in urbanised environments, noise pollution is fast becoming a significant health problem.

As only a portion of this interview was included within the Alder Hey piece, I thought it might be interesting to share some of the additional material. The interview was recorded at FACT in Liverpool, back in April of last year and explores some of the causes and concerns towards noise in the modern world.

Additional info

  • Further reading on the health effects of noise: a WHO report on the burden of disease from environmental noise
  • Nature on BBC Radio 4 is recommended listening if you want to hear more of Chris and his stunning wildlife recordings.
  • Touch Music also releases sound work by Chris, you can browse his collection here.

Audio piece: Winter’s Rest

This is a short audio piece which was originally created for the In The Dark Christmas party – however it didn’t quite fit the bill in the end, being as it is, devoid of any Christmas cheer. Anyway, I thought I’d better not let it go to waste and instead make it available on here – I suppose this is the obligatory Christmas themed post.

Image: youngdoo (Flickr)

It’s another short piece (see Adam as machine) which has fallen out of The D-Word, a documentary I produced over the summer which will be appearing on Transom.org early in the new year. This piece features pathologist, Dr Stuart Hamilton and was recorded in the mortuary at the Sunderland Royal Infirmary back in July. The material I recorded with Stuart at the mortuary only forms a small part of the overall documentary, yet I think it’s interesting enough to justify an entire piece on it’s own, maybe I’ll get around to it one day.

Dr Hamilton explained to me how winter was a particularly busy period for the mortuary staff, with mortality rates increasing in the elderly over the colder months of the year. Another interesting point was that they tended to receive an increase in the number of decomposed bodies at Christmas, but I’ll let you listen to the piece to find out why…

I’m not going to go into a massive rant on how important it is to make an effort to spend time with family, because I’m particularly guilty of not doing so. It just seems that the mortuary staff gain a depressing insight into the mistakes we make and how we choose to lead our lives.

 

Birdsong: An Escape From The Noise

Image:_John Brown (Flickr)

I was interested to read today about a new research project being undertaken, to investigate the psychological impacts of exposure to birdsong. In particular the project will look at how birdsong affects our psychological state, including its effect on mood, attention and sense of creativity.

The research is being conducted as a joint collaboration between the University of Surrey, National Trust and Surrey Wildlife Trust. Researcher Eleanor Ratcliffe, highlighted that there was a real a lack of evidence on the effects of birdsong, stating:

“A great deal of anecdotal evidence suggests that we respond positively to birdsong. However, currently there is a lack of scientific research on the psychological effects of listening to birds.”

You can find out more about the project here.

For me the sound of birdsong offers predominantly positive associations. Living in London, I’m now surrounded by a largely synthetic soundscape, which is strongly connected to the stresses and frustrations of city life (the daily commute, working long hours and a persistent sense of fatigue).

Living amongst this hubbub has unsurprisingly increased the value I attribute to natural soundscapes. Standing in binary opposition to the din of urban living, natural soundscapes offer potential for escape, not just from noise, but from all the negative associations paired with it.

It may be that natural sounds can help us escape from a chaotic lifestyle or at least provide a restorative effect from stress. Understanding the psychological impacts of birdsong will allow us to better understand how we respond to such sounds and perhaps learn more about this relationship. If birdsong really does improve our state of mind and / or sense of wellbeing then it could have real potential in it’s application as a therapeutic tool.

Birdsong as a therapeutic tool?

Back in April / May – I produced a radio piece which looked at the use of Birdsong in the healthcare environment. Alder Hey Children’s Hospital in Liverpool has been experimenting with the use of birdsong to improve the experiences of it’s young patients.

Installed in the central corridor is a sound installation playing the beautiful birdsong recordings made by Chris Watson and Alder Hey patients. These recordings are also used with patients during traumatic and painful procedures, often as a way of calming them down or taking their minds off the situation.

Speaking to the hospital’s Arts Coordinator Vicky Charnock, I found out that there was already tremendous anecdotal evidence in support of birdsong as a therapeutic tool. They were also interested in setting up some form of trial in which to test the potential therapeutic benefits of birdsong.

You can listen to and download the piece here:

The Joy of Birds

Finally figured out how to embedd this video…

This is a short video I produced last month with Nisha Ligon, Tom Welch and Camilla Ruz for the Guardian website. It features BBC wildlife presenter Kate Humble and Martin McGill who were promoting their new book ‘Watching Waterbirds with Kate Humble‘ at the London Wetlands Centre.

We got really lucky with the weather. Turning up at Hammersmith station in the morning we were greeted with a torrential downpour – not ideal for shooting wildlife outside. Thankfully the rain subsided just as we arrived at the Wetlands Centre and we were granted with an afternoon’s worth of sunshine to walk round the site and capture some of the wildlife on camera.

Check out the video below to have a look for yourself!

Watch here: http://www.guardian.co.uk/environment/video/2011/aug/31/kate-humble-birdwatching-london-wetland-video

… or below if it works:

Vodpod videos no longer available.

The London Wetlands Centre is an unusual oasis of wildlife and greenery which is situated surprisingly within the city confines near Hammersmith. It provides people with a great chance to break free from the usual urban surroundings and take in some of the natural scenery usually reserved to those living outside the city limits.

GO SEE.

http://www.wwt.org.uk/visit-us/london

Audio tour: Hospital mortuary

Continuing with the rather morbid theme is another short audio piece based around a hospital mortuary, taken from my ongoing radio project which is probing contemporary attitudes to death.

What’s interesting is that mortuaries are notoriously hard to find within hospitals, they’re rarely signposted and often kept out of sight from patients and staff. It is almost as if death, through an act of denial, is banished from all walks of hospital life, forcing it underground where it remains out of sight and out of mind.

This is perhaps understandable when we consider the way in which modern medicine deals with death. So often is it concerned with prolonging life and keeping a patient alive that death has come to be seen as ‘failure’ – the inability to sustain life. As such death represents medicine’s limits, it reminds us of our own mortality and as such is buried away within the hospital. This isn’t just medicine’s fault, it’s a product of society and our own collective inability to accept death, we seem to live very much in denial, avoiding the ‘d’ word at all costs.

As a result those who work in hospital mortuaries take on an almost ghost like or gothic quality, whose activities remain in the shadows and unknown to the majority who dwell in the wards above. So what exactly goes on in a hospital mortuary? What do these places look like and who would we commonly meet in such a place?

It is perhaps only through the heavily stylised perspective of television and film that we gain any sort of insight into such places. Programmes such as CSI and Silent Witness are famous for their representation of pathology and post-mortem procedures, but they depict a world which is somewhat detached from the reality of such work. These programmes are slick, full of drama and entertaining to watch; they do little to challenge our relationship with death and present a very skewed image of those who work with the deceased. Instead they seem to maintain a common perception that death is something quite perverse, that mortuaries are dark gothic places and that pathologists run around solving murders (and of course that all bodies end up in the mortuary because of hideous murders).

Obviously these programmes shouldn’t to be re-written to reflect reality (probably wouldn’t make for very entertaining viewing), but it’s interesting none the less to look at how they deal with death and the contrast between fiction and reality.

What exactly goes on inside a hospital mortuary?

So the hospital mortuary remains a place largely unknown and unless you were unfortunate enough to visit one (dead or alive), it’s unlikely that you’d ever get a chance to peer inside. I certainly had no idea of what to expect when I recently visited the mortuary at Sunderland Royal Hospital. In the end it was perhaps not surprising to see just how normal mortuary life was, from sitting around drinking tea in a staff room to the usual ‘office banter’ being thrown about as the day went on. Pathologist Dr Stuart Hamilton was my guide for the day and he very kindly showed me round, you can listen to a brief audio tour below:

Adam as machine

Audio piece

I’m currently in the midst of producing a 30 minute radio piece on the subject of death. To be more specific, I’m exploring contemporary attitudes towards death and doing so through the perspective of those who deal with death on a daily basis.

Part of my project took me to the basement of the Royal Sunderland Hospital where I spent the day exploring it’s mortuary with pathologist Dr Stuart Hamilton. Recently, as I was listening back to his interview I begun to find myself fixated on a particular section, in which he discusses the way in which he views the human body. He describes his view of life as being very ‘mechanistic’ and as I listened to him talk about the body as ‘pumps’ and ‘shunts’ I was inspired to compose a short piece from his words.

Adam and Eve; the body electric. Image: Mike Reedy

As a biologist myself, I have always considered the human body to represent a beautiful feat of natural engineering. From the minute intricacies of the inner ear to the extent of the circulatory system, pulsating to the rhythm of a beating heart – the human body is living machine. Our consciousness and everything that makes us, ‘us’ is a product of this machinery and when the machine stops – so do we.

The title of this piece toys with the idea of Adam and Eve and how this throws up a very different view of the human body. However which ever view you take, both are unified in the fact that they find certain beauty in the human form.

Soundcloud’s audio compression has reduced the quality of the piece somewhat so please get in touch if you’d like a copy of the original file.

Putting the Higgs on paper

Toya's colourful portrait of Murray Gell-Mann. Image: Toya Walker

Physicist Lily Asquith recently wrote an interesting and accessible piece on quarks, which was featured on Jon Butterworth’s excellent Life and Physics blog. Included within the post was an interesting depiction of physicist Murray Gell-Mann, as provided by illustrator and artist Toya Walker. Toya also happens to be the individual behind the imaginative and colourful imagery associated with the LHCsound project  – many of which are used to help explain some of the complex concepts behind the process of sonification and particle physics.

When attempting to communicate complicated ideas and concepts we often turn to visual aids to help translate information into an easily digestible format. We perceive the world from a predominantly visual perspective, meaning that illustrations and diagrams are an often more effective mode of communication, especially when dealing with higher levels of complexity. For example, an annotated diagram of the heart is often much easier to understand than a detailed description of it’s anatomy. However such ‘diagrams’ can be uninspiring and do little to engage with those not already interested or familiar with the subject concerned.

Visual aid: Toya's images help to explain how the project works. Image: Toya Walker

The use of visual metaphor and analogy can thus be very helpful in reaching out to a wider audience; for example one of Toya’s images depicts the Higgs Boson as the ‘Golden Snitch’ (as found in Harry Potter) in a humorous and accessible reference to the elusive nature of the particle. As such visual imagery is tasked with the responsibility of translating specialist information into a language that is easily comprehendible but also appealing to non-specialist audiences. This is what I appreciate in Toya’s work. Even in a project that is primarily concerned with conveying information through sound, there is still significant emphasis dedicated to the use of visual imagery to explain key concepts. As a result, the project opens up the Large Hadron Collider (and the excitement associated with it), to those who may have once been alienated by it’s complexities (like me!).

I obviously really like the work Toya has produced for the project, so I got in touch with her to find out a little bit more about how she got involved and developed her illustrations. You can check out her website and blog to have a look at of her artwork, but in the mean time she was kind enough to answer some of my questions and provide some early sketches; you can read her responses below:

Do you have a background in science?

I always enjoyed science at school and took physics at A-level, but not beyond that.

How did you come to work on the LHCsound project?

I met the initiator of the project, physicist Dr. Lily Asquith. The project, of course, was fascinating but what really made me want to get involved was Lily’s passion for communicating the ideas of the project, and indeed the work going on at the LHC, with as large and diverse an audience as possible.

Early sketches toying with the idea of the ATLAS detector inside a music box. Image: Toya Walker

Did your perception and understanding of the LHC change as you developed this work?

Yes, definitely, I did quite a lot of reading to try and understand as fully as I could some of the concepts we were trying to communicate. Learning more really ignited my sense of wonder, its almost like magic, but real. I think physics is often seen as difficult and therefore dull, part of the reason for doing the project was to communicate how exciting it actually is.

A lot of the science and ideas associated with the LHC are quite abstract in nature; did this make it easier or more difficult to create a visual aesthetic? 

A great deal of my working practice involves images drawn from observation so the abstract nature is definitely challenging, but in some ways liberating too. With the project, there were so many approaches that sprang to mind, the real difficulty was choosing a direction. I felt my role was to realize the artwork in the way Lily would visualize it, as she has a very unique and approachable way of talking about particle physics.

In one of your images you’ve depicted the Higgs Boson as the ‘Golden Snitch’ (from Harry Potter), what was the reason for this?

The ATLAS detector is depicted as a music box with the Higgs Boson escaping as the Golden Snitch. Image: Toya Walker

To add humour, I guess, and a reference that might communicate the idea of the search for the elusive Higgs in an accessible way.

Your work in this project has been important in helping to communicate complex ideas in a simpler manner – how do you think art and illustration can help communicate complex ideas and allow people to engage with them?

I think its fundamental, particularly if the goal is to communicate to as large an audience as possible. I’m involved in education work, including the Picture It project run by The House of Illustration. The aim of the project is to use illustration as a learning tool in the classroom for cross-curricular benefit. They’ve done some wonderful work including sessions where primary school children made their own books, with every page showing a different stage of the process of a volcano erupting. Or secondary students used collage to show the effects of different forces. I think it’s such a brilliant and effective way of learning.

In science information is often communicated graphically in visual form. Scientists would often see this as an objective representation of information – but as an artist do you think such representation can take on an additional expressive form?

I find data visualization really interesting and I often look at the work at http://www.informationisbeautiful.net/. There are brilliant ways of communicating vast amounts of information.  None of the imagery made for LHCsound was trying to communicate data, just ideas and we definitely made a conscious effort to develop the pictures in a different way, or a style not traditionally associated with science textbooks for instance.

Toya Walker is an illustrator and artist who lives and works in South London. A graduate of Edinburgh College of Art she has an eclectic approach to image making but drawing and painting form the basis for all her work. Her illustrations have been used by The  Viral Factory, Atlantic Records and the LHCsound project and work has been shortlisted for The Jerwood Drawing Prize, The Dulwich Picture Gallery Summer Exhibition and The Mall Galleries Royal Society shows.