Enhancing pre-employment transition services: A type 1 hybrid randomized controlled trial protocol for evaluating WorkChat: A Virtual Workday among autistic transition-age youth.

Autistic transition-age youth experience high rates of unemployment and underemployment, in part due to the social challenges they may face when having conversations in the workplace. In an effort to help enhance conversational abilities in the workplace, our collaborative team partnered to develop WorkChat: A Virtual Workday. Specifically, our team of scientists, community partners, and diversity and inclusion experts participated in a community-engaged process to develop WorkChat using iterative feedback from autistic transition-age youth and their teachers. With initial development complete, this study reports on the protocol that our collaborative team developed, reviewed, and approved to conduct a randomized controlled trial (RCT) to evaluate the real-world effectiveness and initial implementation process outcomes of WorkChat when integrated into post-secondary pre-employment transition services (Pre-ETS). Our aims are to: 1) evaluate whether services-as-usual in combination with WorkChat, compared to services-as-usual with an attention control, enhances social cognition and work-based social ability (between pre- and post-test); reduces anxiety about work-based social encounters (between pre- and post-test), and increases sustained employment by 9-month follow-up; 2) evaluate whether social cognitive ability and work-based social ability mediate the effect of WorkChat on sustained employment; and 3) conduct a multilevel, mixed-method process evaluation of WorkChat implementation.

Mechanism of Action for Obtaining Job Offers With Virtual Reality Job Interview Training.

OBJECTIVE: Four randomized controlled trials revealed that virtual-reality job interview training (VR-JIT) improved interviewing skills and the odds of obtaining a job offer among trainees with severe mental illness or autism spectrum disorder. This study assessed whether postintervention interviewing skills mediated the relationship between completion of virtual interviews and receiving job offers by six-month follow-up. METHODS: VR-JIT trainees (N=79) completed pre- and posttest mock interviews and a brief survey approximately six months later to assess whether they received a job offer. RESULTS: As hypothesized, analyses indicated that the number of completed virtual interviews predicted greater posttest interviewing skills (ß=.20, 95% posterior credible interval [PCI]=.08-.33), which in turn predicted trainees' obtaining a job offer (ß=.28, 95% PCI=.01-.53). CONCLUSIONS: VR-JIT may provide a mechanism of action that helps trainees with various psychiatric diagnoses obtain job offers in the community. Future research can evaluate the community-based effectiveness of this novel intervention.

Virtual Reality Job Interview Training and 6-Month Employment Outcomes for Individuals with Substance Use Disorders Seeking Employment.

BACKGROUND: Individuals with substance use disorders (SUDs) have low employment rates and job interviewing is a critical barrier to employment for them. Virtual reality training is efficacious at improving interview skills and vocational outcomes for several clinical populations. OBJECTIVE: This study evaluated the acceptability and efficacy of virtual reality job interview training (VR-JIT) at improving interview skills and vocational outcomes among individuals with SUDs via a small randomized controlled trial (n=14 VR-JIT trainees, n=11 treatment-as-usual (TAU) controls). METHODS: Trainees completed up to 10 hours of virtual interviews, while controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews and vocational outcomes at six-month follow-up. RESULTS: Trainees reported that the intervention was easy-to-use and helped prepared them for future interviews. While co-varying for pre-test role-play performance, trainees had higher post-test role-play scores than controls at the trend level (p<0.10). At 6-month follow-up, trainees were more likely than controls to attain a competitive position (78.6% vs. 44.4%, p<0.05, respectively). Trainees had greater odds of attaining a competitive position by 6 month follow-up compared to controls (OR: 5.67, p<0.05). VR-JIT participation was associated with fewer weeks searching for a position (r= -0.36, p<0.05). CONCLUSIONS: There is preliminary evidence that VR-JIT is acceptable to trainees. Moreover, VR-JIT led to better vocational outcomes with trainees having greater odds of attaining a competitive position by 6-month follow-up. Future studies could evaluate the effectiveness of VR-JIT within community-based services.

Job Offers to Individuals With Severe Mental Illness After Participation in Virtual Reality Job Interview Training.

OBJECTIVE: Individuals with severe mental illness have low employment rates, and the job interview presents a critical barrier for them to obtain competitive employment. Prior randomized controlled trials (RCTs) indicated that virtual reality job interview training (VR-JIT) improved job interviewing skills among trainees. This study assessed whether VR-JIT participation was associated with greater odds of receiving job offers in the six-months after completion of training. METHODS: To assess the efficacy of VR-JIT, trainees (N=39) in the method and a comparison group (N=12) completed a brief survey approximately six months after participating in the RCTs. Primary vocational outcome measures included receiving a job offer and number of weeks searching for employment. RESULTS: A larger proportion of trainees than comparison participants received a job offer (51% versus 25%, respectively). Trainees were more likely to receive a job offer than comparison participants (odds ratio=9.64, p=.02) after analyses accounted for cognition, recency of last job, and diagnosis. Trainees had greater odds of receiving a job offer for each completed VR-JIT trial (odds ratio=1.41, p=.04), and a greater number of completed VR-JIT trials predicted fewer weeks of searching for employment (ß=-.74, p=.02). CONCLUSIONS: Results provide preliminary support that VR-JIT is a promising intervention associated with enhanced vocational outcomes among individuals with severe mental illness. Given that participants had minimal access to standardized vocational services, future research could evaluate the effectiveness of VR-JIT among individuals with and without access to standardized vocational services as well as evaluate strategies to implement VR-JIT within a large community mental health service provider.

Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment.

BACKGROUND: Individuals with schizophrenia have low employment rates and the job interview presents a critical barrier for them to obtain employment. Virtual reality training has demonstrated efficacy at improving interview skills and employment outcomes among multiple clinical populations. However, the effects of this training on individuals with schizophrenia are unknown. This study evaluated the efficacy of virtual reality job interview training (VR-JIT) at improving job interview skills and employment outcomes among individuals with schizophrenia in a small randomized controlled trial (n=21 VR-JIT trainees, n=11 waitlist controls). METHODS: Trainees completed up to 10h of virtual interviews using VR-JIT, while controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews scored by blinded raters with expertise in human resources and self-reported interviewing self-confidence. Six-month follow-up data on employment outcomes were collected. RESULTS: Trainees reported that the intervention was easy-to-use, helpful, and prepared them for future interviews. Trainees demonstrated increased role-play scores between pre-test and post-test while controls did not (p=0.001). After accounting for neurocognition and months since prior employment, trainees had greater odds of receiving a job offer by 6month follow-up compared to controls (OR: 8.73, p=0.04) and more training was associated with fewer weeks until receiving a job offer (r=-0.63, p<0.001). CONCLUSIONS: Results suggest that VR-JIT is acceptable to trainees and may be efficacious for improving job interview skills in individuals with schizophrenia. Moreover, trainees had greater odds of receiving a job offer by 6-month follow-up. Future studies could evaluate the effectiveness of VR-JIT within community-based services.

Brief report: vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training.

Young adults with high-functioning autism spectrum disorder (ASD) have low employment rates and job interviewing presents a critical barrier to employment for them. Results from a prior randomized controlled efficacy trial suggested virtual reality job interview training (VR-JIT) improved interviewing skills among trainees with ASD, but not controls with ASD. We conducted a brief survey with 23 of 26 participants from this study to evaluate their vocational outcomes at 6-month follow-up with a focus on whether or not they attained a competitive position (employment or competitive volunteering). Logistic regression indicated VR-JIT trainees had greater odds of attaining a competitive position than controls (OR 7.82, p < 0.05). Initial evidence suggests VR-JIT is a promising intervention that enhances vocational outcomes among young adults with high-functioning ASD.

Virtual Reality Job Interview Training for Veterans with Posttraumatic Stress Disorder.

BACKGROUND: Veterans with posttraumatic stress disorder (PTSD) have low employment rates and the job interview presents a critical barrier for them to obtain competitive employment. OBJECTIVE: To evaluate the acceptability and efficacy of virtual reality job interview training (VR-JIT) among veterans with PTSD via a small randomized controlled trial (n=23 VR-JIT trainees, n=10 waitlist treatment-as-usual (TAU) controls). METHODS: VR-JIT trainees completed up to 10 hours of simulated job interviews and reviewed information and tips about job interviewing, while wait-list TAU controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews scored by blinded human resource experts and self-reported interviewing self-confidence. RESULTS: Trainees attended 95% of lab-based VR-JIT sessions and found the intervention easy-to-use, helpful, and prepared them for future interviews. VR-JIT trainees demonstrated significantly greater improvement on role-play interviews compared with wait-list TAU controls (p=0.04) and demonstrated a large effect for within-subject change (Cohen's d=0.76). VR-JIT performance scores increased significantly over time (R-Squared=0.76). Although VR-JIT trainees showed a moderate effect for within-subject change on self-confidence (Cohen's d=0.58), the observed difference between conditions did not reach significance (p=0.09). CONCLUSIONS: Results provide preliminary support that VR-JIT is acceptable to trainees and may be efficacious for improving job interview skills and self-confidence in veterans with PTSD.

Virtual reality job interview training for individuals with psychiatric disabilities.

Services are available to help support existing employment for individuals with psychiatric disabilities; however, there is a gap in services targeting job interview skills that can help obtain employment. We assessed the feasibility and efficacy of Virtual Reality Job Interview Training (VR-JIT) in a randomized controlled trial. Participants were randomized to VR-JIT (n = 25) or treatment-as-usual (TAU) (n = 12) groups. VR-JIT consisted of 10 hours of simulated job interviews with a virtual character and didactic online training. The participants attended 95% of laboratory-based training sessions and found VR-JIT easy to use and felt prepared for future interviews. The VR-JIT group improved their job interview role-play performance (p ≤ 0.05) and self-confidence (p ≤ 0.05) between baseline and follow-up as compared with the TAU group. VR-JIT performance scores increased over time (R = 0.65). VR-JIT demonstrated initial feasibility and efficacy at improving job interview skills and self-confidence. Future research may help clarify whether this intervention is efficacious in community-based settings.

Simulated job interview improves skills for adults with serious mental illnesses.

Adults with serious mental illnesses (e.g., Autism Spectrum Disorder [ASD], schizophrenia, post-traumatic stress disorder [PTSD]) often have difficulties obtaining employment. The Job Interview Training System with Molly Porter, developed in collaboration with Yale and Northwestern Universities and vocational rehabilitation specialists with funding from The National Institutes of Health (R43/44MH080496), allows learners to practice job interviews on computers in a stress free environment. The system includes user-driven educational materials, an interactive job application, a practice simulation with a fictional interviewer (Molly Porter), and extensive feedback. SIMmersion's PeopleSIM™ technology allows each conversation with Molly to provide a unique interview experience, enabling users to gain confidence while building skills. The on-screen coach provides insight during the conversation, and a comprehensive after-action review provides learners with feedback on the entire interview. In a randomized control trial, the system was proven effective at improving participants' interview skills and confidence. Ninety-six (96) unemployed adults with ASD (n=26), schizophrenia/other (n=37) or PTSD (n=33) were recruited. Participants were randomized into control (n=32) and experimental (n=64) conditions. The control group was "wait-listed" to receive training, and the experimental group used the training system with Molly Porter. Both groups completed pre- and post-intervention role-play interviews and self-assessment questionnaires. Analyses of covariance showed that the simulation provided a highly significant training effect, with experimental group participants scoring better in the role-play interviews and self-assessing higher than control group participants. By increasing skills and confidence, this system may ultimately reduce the length of unemployment for adults with mental illnesses.

Virtual reality job interview training in adults with autism spectrum disorder.

The feasibility and efficacy of virtual reality job interview training (VR-JIT) was assessed in a single-blinded randomized controlled trial. Adults with autism spectrum disorder were randomized to VR-JIT (n = 16) or treatment-as-usual (TAU) (n = 10) groups. VR-JIT consisted of simulated job interviews with a virtual character and didactic training. Participants attended 90 % of laboratory-based training sessions, found VR-JIT easy to use and enjoyable, and they felt prepared for future interviews. VR-JIT participants had greater improvement during live standardized job interview role-play performances than TAU participants (p = 0.046). A similar pattern was observed for self-reported self-confidence at a trend level (p = 0.060). VR-JIT simulation performance scores increased over time (R(2) = 0.83). Results indicate preliminary support for the feasibility and efficacy of VR-JIT, which can be administered using computer software or via the internet.

Virtual conversation partner for adults with autism.

Autistic Spectrum Disorder (ASD) is notable for severely impaired reciprocal social interaction skills relative to language and intellectual abilities, presenting a major barrier to social integration and vocational success. Evidence-based interventions to address these needs are lacking. We report on the development of a small, prototype conversation simulation to teach conversational skills to adolescents and adults with ASD and average to superior intellectual abilities. We also report on a test of the feasibility and acceptability of the simulation approach with a sample of the target population. The simulation engages the user in a virtual conversation with an on-screen partner whose reactions provide naturalistic feedback geared to the appropriateness of the learner's response choices. The prototype simulation, which provides for up to 12 potentially unique multi-turn conversations, was used over a period of 2 weeks by 16 adolescents and adults who then rated statements about the system on a linear scale of 1 (disagreement) to 5 (high agreement). The participants highly endorsed the majority of positive statements about the quality and credibility of the interaction and the virtual conversation partner. In contrast, agreement with positive statements about instructional features external to the conversation was moderate. Unexpectedly, most participants strongly agreed that using the simulation had been helpful to them. Further development and testing in the context of a controlled study with randomized assignment to control and experimental groups are needed to determine whether this approach is effective in improving real-world pragmatic language behavior of high-functioning adults with ASD.

Virtual reality skills training for health care professionals in alcohol screening and brief intervention.

BACKGROUND: Educating physicians and other health care professionals about the identification and treatment of patients who drink more than recommended limits is an ongoing challenge. METHODS: An educational randomized controlled trial was conducted to test the ability of a stand-alone training simulation to improve the clinical skills of health care professionals in alcohol screening and intervention. The "virtual reality simulation" combined video, voice recognition, and nonbranching logic to create an interactive environment that allowed trainees to encounter complex social cues and realistic interpersonal exchanges. The simulation included 707 questions and statements and 1207 simulated patient responses. RESULTS: A sample of 102 health care professionals (10 physicians; 30 physician assistants or nurse practitioners; 36 medical students; 26 pharmacy, physican assistant, or nurse practitioner students) were randomly assigned to a no training group (n = 51) or a computer-based virtual reality intervention (n = 51). Professionals in both groups had similar pretest standardized patient alcohol screening skill scores: 53.2 (experimental) vs 54.4 (controls), 52.2 vs 53.7 alcohol brief intervention skills, and 42.9 vs 43.5 alcohol referral skills. After repeated practice with the simulation there were significant increases in the scores of the experimental group at 6 months after randomization compared with the control group for the screening (67.7 vs 58.1; P < .001) and brief intervention (58.3 vs 51.6; P < .04) scenarios. CONCLUSIONS: The technology tested in this trial is the first virtual reality simulation to demonstrate an increase in the alcohol screening and brief intervention skills of health care professionals.

Virtual standardized patients for training health professionals on alcohol screening and brief intervention.

Alcohol use disorders are common, but are often not discussed with patients. Through NIAAA funding, SIMmersion LLC and the University of Wisconsin are producing an interactive, immersive computer-based simulation that will allow learners to conduct an alcohol screening and brief intervention with a virtual standardized patient. The validity of this simulation will be evaluated through a study involving 100 clinicians.

Simulation and virtual reality in medical education and therapy: a protocol.

Continuing medical education has historically been provided primarily by didactic lectures, though adult learners prefer experiential or self-directed learning. Young physicians have extensive experience with computer-based or "video" games, priming them for medical education--and treating their patients--via new technologies. We report our use of standardized patients (SPs) to educate physicians on the diagnosis and treatment of biological and chemical warfare agent exposure. We trained professional actors to serve as SPs representing exposure to biological agents such as anthrax and smallpox. We rotated workshop participants through teaching stations to interview, examine, diagnose and treat SPs. We also trained SPs to simulate a chemical mass casualty (MASCAL) incident. Workshop participants worked together to treat MASCAL victims, followed by discussion of key teaching points. More recently, we developed computer-based simulation (CBS) modules of patients exposed to biological agents. We compare the strengths and weaknesses of CBS vs. live SPs. Finally, we detail plans for a randomized controlled trial to assess the efficacy of virtual reality (VR) exposure therapy compared to pharmacotherapy for post-traumatic stress disorder (PTSD). PTSD is associated with significant disability and healthcare costs, which may be ameliorated by the identification of more effective therapy.

Interactive simulation training: computer simulated standardized patients for medical diagnosis.

SIMmersion LLC is a software development company that creates human interactive training simulations. SIMmersion produces PC-based simulations of people with whom trainees are able to hold conversations. Trainees communicate with simulated characters in face-to-face conversations, using voice recognition technology. Topics that would match well with SIMmersion's simulation capabilities include diagnosis of depression, alcoholism, or drug use, grievance counseling, marriage counseling, suicide intervention, and first responder training for biological terrorism. SIMmersion's technology could be integrated with modeling technology to create a speech-interactive, mannequin-based patient simulation system for healthcare training and assessment. Systems currently being developed involve early detection of smallpox as a result of a biological attack.