How Brain Scans Can be Used to See What a Person Was Reading
By analysing MRI images of the brain with a special mathematical model, it is possible to reconstruct thoughts more accurately than ever before. In this way, researchers from Radboud University Nijmegen have succeeded in determining from brain-scans which letter a test subject was looking at.
Functional MRI scanners have been used in cognition research primarily to determine which brain areas are active while test subjects perform a specific task. The question is simple: is a particular brain region on or off? A research group at the Donders Institute for Brain, Cognition and Behaviour at Radboud University in the Netherlands has gone a step further: they have used data from the scanner to determine what a test subject is looking at. The researchers ‘taught' a model how small volumes of 2x2x2 mm from the brain scans - known as voxels - respond to individual pixels. By combining all the information about the pixels from the voxels, it became possible to reconstruct the image, in this case hand-written letters, viewed by the subject. The result was not a clear image, but a somewhat fuzzy speckle pattern.
Prior knowledge improves model performance
‘After this we did something new', says lead researcher Marcel van Gerven. ‘We gave the model prior knowledge: we taught it what letters look like. This improved the recognition of the letters enormously. The model compares the letters to determine which one corresponds most exactly with the speckle image, and then pushes the results of the image towards that letter. The result was the actual letter, a true reconstruction.'
Improved resolution; more possibilities
‘In our further research we will be working with a more powerful MRI scanner,' explains Sanne Schoenmakers, who is working on a thesis about decoding thoughts. ‘Due to the higher resolution of the scanner, we hope to be able to link the model to more detailed images. We are currently linking images of letters to 1200 voxels in the brain; with the more powerful scanner we will link images of faces to 15,000 voxels.' -- Radboud University Nijmegen
Reference:
S. Schoenmakers, M. Barth, T.Heskes, M.A.J. van Gerven, Linear reconstruction of perceived images from human brain activity, Neuroimage, 2013, DOI: 10.1016/j.neuroimage.2013.07.043
See Now: NASA's Juno Spacecraft's Rendezvous With Jupiter's Mammoth Cyclone
Join the Conversation