Brain-reading uses the responses of multiple voxels in the brain evoked by stimulus then detected by fMRI in order to decode the original stimulus. Brain reading studies differ in the type of decoding (i.e. classification, identification and reconstruction) employed, the target (i.e. decoding visual patterns, auditory patterns, cognitive states), and the decoding algorithms (linear classification, nonlinear classification, direct reconstruction, Bayesian reconstruction, etc.) employed.
Contents
Classification
In classification, a pattern of activity across multiple voxels is used to determine the particular class from which the stimulus was drawn. Many studies have classified visual stimuli, but this approach has also been used to classify cognitive states.
Reconstruction
In reconstruction brain reading the aim is to create a literal picture of the image that was presented. Early studies used voxels from early visual cortex areas (V1, V2, and V3) to reconstruct geometric stimuli made up of flickering checkerboard patterns.
Natural images
More recent studies used voxels from early and anterior visual cortex areas forward of them (visual areas V3A, V3B, V4, and the lateral occipital) together with Bayesian inference techniques to reconstruct complex natural images. This brain reading approach uses three components: A structural encoding model that characterizes responses in early visual areas; a semantic encoding model that characterizes responses in anterior visual areas; and a Bayseian prior that describes the distribution of structural and semantic scene statistics.
Experimentally the procedure is for subjects to view 1750 black and white natural images that are correlated with voxel activation in their brains. Then subjects viewed another 120 novel target images, and information from the earlier scans is used reconstruct them. Natural images used include pictures of a seaside cafe and harbor, performers on a stage, and dense foliage.
Other types
It is possible to track which of two forms of rivalrous binocular illusions a person was subjectively experiencing from fMRI signals. The category of event which a person freely recalls can be identified from fMRI before they say what they remembered. Statistical analysis of EEG brainwaves has been claimed to allow the recognition of phonemes, and at a 60% to 75% level color and visual shape words. It has also been shown that brain-reading can be achieved in a complex virtual environment.
Accuracy
Brain-reading accuracy is increasing steadily as the quality of the data and the complexity of the decoding algorithms improve. In one recent experiment it was possible to identify which single image was being seen from a set of 120. In another it was possible to correctly identify 90% of the time which of two categories the stimulus came and the specific semantic category (out of 23) of the target image 40% of the time.
Limitations
It has been noted that so far brain-reading is limited. "In practice, exact reconstructions are impossible to achieve by any reconstruction algorithm on the basis of brain activity signals acquired by fMRI. This is because all reconstructions will inevitably be limited by inaccuracies in the encoding models and noise in the measured signals. Our results demonstrate that the natural image prior is a powerful (if unconventional) tool for mitigating the effects of these fundamental limitations. A natural image prior with only six million images is sufficient to produce reconstructions that are structurally and semantically similar to a target image."
Applications
Brain-reading has been suggested as an alternative to polygraph machines as a form of lie detection. Another alternative to polygraph machines is blood oxygenated level dependent functional MRI technology (BOLD fMRI). This technique involves the interpretation of the local change in the concentration of oxygenated hemoglobin in the brain, although the relationship between this blood flow and neural activity is not yet completely understood.
A number of concerns have been raised about the accuracy and ethical implications of brain-reading for this purpose. Laboratory studies have found rates of accuracy of up to 85%; however, there are concerns about what this means for false positive results among non-criminal populations: "If the prevalence of "prevaricators" in the group being examined is low, the test will yield far more false-positive than true-positive results; about one person in five will be incorrectly identified by the test." Ethical problems involved in the use of brain-reading as lie detection include misapplications due to adoption of the technology before its reliability and validity can be properly assessed and due to misunderstanding of the technology, and privacy concerns due to unprecedented access to individual's private thoughts. However, it has been noted that the use of polygraph lie detection carries similar concerns about the reliability of the results and violation of privacy.
Brain-reading has also been proposed as a method of improving human-machine interfaces, by the use of EEG to detect relevant brain states of a human. In recent years, there has been a rapid increase in patents for technology involved in reading brainwaves, rising from fewer than 400 from 2009–2012 to 1600 in 2014. These include proposed ways to control video games via brain waves and "neuro-marketing" to determine someone's thoughts about a new product or advertisement.