The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is a computerized measure that was developed to be a reliable, valid, and sensitive measure of functional capacity, with the potential to demonstrate real-world functional improvements associated with cognitive change. The VRFCAT presents participants with a realistic simulated environment to recreate routine activities of daily living. The VRFCAT was developed by NeuroCog Trials under National Institute of Mental Health (NIMH) Small Business Innovation Research (SBIR) phase 1 and phase 2 grants.
Contents
History
In an effort to address the unmet need for standard methods in the development of treatments of cognitive impairment associated with schizophrenia, representatives from the US Food and Drug Administration (FDA), NIMH, the pharmaceutical industry, and academia, formed the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) project. The MATRICS group and FDA representatives concluded that cognitive improvements alone, as measured by standardized neuropsychological assessments, were not a sufficient demonstration of drug efficacy to enable an indication for cognitive impairments associated with schizophrenia. The approval of new treatments for cognitive impairments would also require evidence that the cognitive improvements are clinically meaningful. Thus, clinical trials of cognitive enhancing compounds in schizophrenia must demonstrate improvement on a standard cognitive measure and improvement on a co-primary measure indicating that those cognitive improvements have a meaningful impact on skills used in real-world functioning. There are many indicators of real-world functioning in schizophrenia, such as the patient’s ability to hold employment, live independently, and maintain social relationships, but detecting changes may take longer to emerge than the duration of a typical clinical trial. Furthermore, real-world functional change is dependent upon a variety of circumstances unrelated to treatment, such as changes in the patient’s broader milieu. The MATRICS group thus recommended that clinical meaningfulness could be demonstrated through the use of tools measuring the potential to demonstrate real-world functional improvements associated with cognitive change rather than having to demonstrate actual functional change. The MATRICS group made no firm recommendations regarding which measure or measures should be used to provide evidence of a clinically meaningful effect as none of the existing instruments were viewed as sufficiently reliable, valid, and sensitive to treatment interventions during a clinical trial. To meet this need, the VRFCAT was designed to reliably assess functional capacity within the context of a simulated real-world environment.
Description
The VRFCAT uses the Unreal Engine to create a realistic, interactive, and immersive environment consisting of 4 mini-scenarios that follow a story, including: (1) exploring a kitchen, (2) catching a bus to a grocery store, (3) finding/purchasing food in a grocery store, and (4) returning home on a bus. To permit repeated testing while avoiding learning effects, 6 alternate forms were created that vary the ingredients stored in the cabinets, items on the recipe and grocery list, cost of the bus rides, cost of groceries, and money available in a wallet. Subjects complete each scenario through a progressive storyboard design
Validation
In 2014, a validation study was conducted with three aims:
- To assess the validity, sensitivity, and reliability of the VRFCAT as a primary measure of functional capacity in schizophrenia
- To examine the VRFCAT’s ability to quantify changes in functional capacity by comparing it to the UCSD Performance-based Skills Assessment (UPSA-2-VIM)
- To determine the association between performance on the VRFCAT and performance on the MATRICS Consensus Cognitive Battery (MCCB), which is the gold standard measure of cognition in pharmaceutical clinical trials regulated by the FDA
Participants for the study included 158 subjects with schizophrenia, and 166 healthy controls. The VRFCAT demonstrated both good test-retest reliability (ICC = .81), minimal practice effects, and high, positive correlations with cognitive performance (r = .58 with the MCCB). In particular, the VRFCAT Total Time demonstrated good test-retest reliability (ICC = .80 in young adults; ICC = .64 in older adults) and insignificant practice effects; the Total Time variable also had high correlations with cognitive performance across age groups (r = .79 in young adult, r = .66 in older adults). Overall, the VRFCAT was shown to be a measure of functional capacity with good psychometric properties permitting repeated assessment in clinical trials.
Other Cognitive Assessment Tools
SCoRS - Schizophrenia Cognition Rating Scale