Virtual Job Interview Training: A Dose Response to Improve Employment for Transition-Age Youth With Disabilities.

Virtual Interview Training for Transition Age Youth and Virtual Reality Job Interview Training are job interview simulators with demonstrated effectiveness in randomized controlled trials. We evaluated their dose responses via secondary data analysis of 558 transition-age youth with disabilities in 47 schools where the simulators were implemented in quasi-experimental studies. Cut-point analyses determined dosing efficiency and efficacy to optimize competitive employment. The most efficient dose when accounting for the balance between dose and employment was completing nine virtual interviews. The most efficacious dose to maximize the likelihood of successful employment was 38, but varied across race, IQ, IDEA categories and employment history. This study provides a novel approach to inform implementation guidelines for virtual interview training in pre-employment transition services. Limitations and implications for research and practice are discussed.

Implementation Preparation Costs of Virtual Interview Training in Pre-Employment Transition Services: A Budget Impact Analysis.

Virtual Reality Job Interview Training (VR-JIT) and Virtual Interview Training for Transition Age Youth (VIT-TAY) demonstrated initial effectiveness at increasing employment among transition-age youth with disabilities engaged in pre-employment transition services. We characterized activities and estimated the labor and non-labor costs required to prepare schools to implement VR-JIT or VIT-TAY. Implementation preparation and support teams reported labor hours throughout the implementation preparation process. Implementation preparation labor hours at 43 schools cost approximately $1,427 per school, while non-labor costs were $100 per trainee (student). We estimated the replication of implementation preparation labor activities would cost $1,024 per school (range: $841-$1,208). Most costs were spent in delivery planning and teacher training. Given that implementation preparation costs can be barriers to intervention adoption, our results provide critical information for contemplating future implementation of VR-JIT or VIT-TAY.

The Need for Technology-Aided Instruction and Intervention Policy for Autistic Youth.

This paper examines current technology-aided instruction and intervention (TAII) available for autistic transition-age youth (TAY) and existing policies that may support or hinder the delivery of these interventions. Specifically, we focus on policies that might influence the delivery of TAII to autistic TAY. After a careful review of the literature, we observed that postsecondary policy guiding the delivery of TAII designed to support autistic TAY is lacking. TAII have demonstrated effectiveness, usability, sustainability, and cost-effectiveness, particularly with this population. We suggest possibilities for future policies to support the development, implementation, and evaluation of TAII for autistic TAY.

Implementing Virtual Interview Training for Transition-Age Youth: Practical Strategies for Educators.

This column shares strategies for implementing a technology-based intervention within secondary special education transition services. Virtual Interview Training for Transition-Age Youth is a web-based job interview training that uses educational content and virtual hiring managers (Rita and Travis) to help teach job interview skills. Eight strategies were assembled by research staff, teachers, and administrators during two research studies conducted across n=47 schools that delivered the program (commonly called "Rita and Travis") to n=561 students in secondary special education transition programs. Eight support strategies for implementing Rita and Travis are described. Strategies include: prepare for implementation, ensure administrative support, prioritize training and support, address logistical challenges, integrate the program into the curriculum, take advantage of the administrative center, individualize student support, and offer virtual delivery. The purpose of this column is to prepare others for successful implementation of Virtual Interview Training for Transition-Age Youth within special education transition programs.

Psychometric properties of the mock interview rating scale for autistic transition-age youth.

Background: Employment is a major contributor to quality of life. However, autistic people are often unemployed and underemployed. One potential barrier to employment is the job interview. However, the availability of psychometrically-evaluated assessments of job interviewing skills is limited for autism services providers and researchers. Objective: We analyzed the psychometric properties of the Mock Interview Rating Scale that was adapted for research with autistic transition-age youth (A-MIRS; a comprehensive assessment of video-recorded job interview role-play scenarios using anchor-based ratings for 14 scripted job scenarios). Methods: Eighty-five transition-age youth with autism completed one of two randomized controlled trials to test the effectiveness of two interventions focused on job interview skills. All participants completed a single job interview role-play at pre-test that was scored by raters using the A-MIRS. We analyzed the structure of the A-MIRS using classical test theory, which involved conducting both exploratory and confirmatory factor analyzes, Rasch model analysis and calibration techniques. We then assessed internal consistency, inter-rater reliability, and test-retest reliability. Pearson correlations were used to assess the A-MIRS' construct, convergent, divergent, criterion, and predictive validities by comparing it to demographic, clinical, cognitive, work history measures, and employment outcomes. Results: Results revealed an 11-item unidimensional construct with strong internal consistency, inter-rater reliability, and test-retest reliability. Construct [pragmatic social skills (r = 0.61, p < 0.001), self-reported interview skills (r = 0.34, p = 0.001)], divergent [e.g., age (r = -0.13, p = 0.26), race (r = 0.02, p = 0.87)], and predictive validities [competitive employment (r = 0.31, p = 0.03)] received initial support via study correlations, while convergent [e.g., intrinsic motivation (r = 0.32, p = 0.007), job interview anxiety (r = -0.19, p = 0.08)] and criterion [e.g., prior employment (r = 0.22, p = 0.046), current employment (r = 0.21, p = 0.054)] validities were limited. Conclusion: The psychometric properties of the 11-item A-MIRS ranged from strong-to-acceptable, indicating it may have utility as a reliable and valid method for assessing the job interview skills of autistic transition-age youth.

Enhancing pre-employment transition services: A type 1 hybrid randomized controlled trial protocol for evaluating WorkChat: A Virtual Workday among autistic transition-age youth.

Autistic transition-age youth experience high rates of unemployment and underemployment, in part due to the social challenges they may face when having conversations in the workplace. In an effort to help enhance conversational abilities in the workplace, our collaborative team partnered to develop WorkChat: A Virtual Workday. Specifically, our team of scientists, community partners, and diversity and inclusion experts participated in a community-engaged process to develop WorkChat using iterative feedback from autistic transition-age youth and their teachers. With initial development complete, this study reports on the protocol that our collaborative team developed, reviewed, and approved to conduct a randomized controlled trial (RCT) to evaluate the real-world effectiveness and initial implementation process outcomes of WorkChat when integrated into post-secondary pre-employment transition services (Pre-ETS). Our aims are to: 1) evaluate whether services-as-usual in combination with WorkChat, compared to services-as-usual with an attention control, enhances social cognition and work-based social ability (between pre- and post-test); reduces anxiety about work-based social encounters (between pre- and post-test), and increases sustained employment by 9-month follow-up; 2) evaluate whether social cognitive ability and work-based social ability mediate the effect of WorkChat on sustained employment; and 3) conduct a multilevel, mixed-method process evaluation of WorkChat implementation.

Development and Psychometric Properties of Self-Reported Job Interview Skills and Job Interview Anxiety for Autistic Transition-Age Youth.

BACKGROUND: The study of job interview training is an emerging area among transition-age autistic youth who face significant challenges when navigating job interviews. The autism field has limited measures that have undergone rigorous psychometric evaluation. OBJECTIVE: We sought to evaluate the psychometric properties of adapted self-report measures assessing job interview skills and job interview anxiety. METHODS: As part of two parent randomized controlled trials, eighty-five transition-age autistic youth completed measures related to the strength of their job interview skills and their level of job interview anxiety. We conducted classical test theory analyses, exploratory and confirmatory factor analyses, and Rasch model analytic and calibration analyses. Pearson correlations were used to establish concurrent, divergent, and criterion validity by correlating these scales with measures of social differences, depressive symptoms, behaviors, neuropsychological functioning, and work history. RESULTS: Our analyses yielded two brief and reliable scales: Measure of Job Interview Skills (MOJO-iSkills) and Measure of Job Interview Anxiety (MOJO-iAnxiety), which demonstrated initial concurrent, divergent, and criterion validities when correlated with measures of depressive symptoms, social differences, internalizing and externalizing behavior, and work history. CONCLUSIONS: This study presents initial evidence that MOJO-iSkills and MOJO-iAnxiety have acceptable psychometric properties supporting they can be used to reliably and validly assess job interview skills and interview anxiety.

Mixed methods implementation evaluation of virtual interview training for transition-age autistic youth in pre-employment transition services.

BACKGROUND: Autistic transition-age youth are employed at rates far lower than their non-disabled peers as well as youth with other disabilities. Meanwhile, very few studies have evaluated the implementation of job interviewing practices within pre-employment transition services. OBJECTIVE: We conducted an initial implementation evaluation as part of a Type I hybrid randomized controlled effectiveness-implementation trial where we trained teachers to deliver Virtual Interview Training for Transition-Age Youth (VIT-TAY) within five pre-employment transition services programs. METHODS: We used mixed methods to evaluate leader (n=5), teacher (n=15) and autistic transition age youth (n=48) perceptions of VIT-TAY. We used descriptive statistics and thematic network analysis to evaluate survey data. Mixed methods integration was then performed to make comparisons between quantitative and qualitative results. RESULTS: Quantitative survey data revealed that leaders and teachers found VIT-TAY to be highly acceptable and appropriate for pre-employment transition services; findings which were confirmed via thematic network analysis of qualitative interview data. Autistic students reported via quantitative surveys that VIT-TAY was acceptable and usable, which was confirmed via thematic network analysis of open-ended survey data. CONCLUSIONS: This initial implementation evaluation can be used to inform a larger scale implementation evaluation of VIT-TAY in schools.

Scaling Out Virtual Interview Training for Transition-Age Youth: A Quasi-Experimental Hybrid Effectiveness-Implementation Study.

In a previous randomized controlled trial (RCT), Virtual Interview Training for Transition Age Youth (VIT-TAY) enhanced employment for autistic youth receiving transition services. For this study, a non-randomized hybrid effectiveness-implementation trial evaluated whether VIT-TAY maintained its core implementation and effectiveness functions when scaled out to students with disabilities in 32 schools. Also, we compared the implementation and effectiveness between VIT-TAY and Virtual Reality Job Interview Training (VR-JIT; developed for adults and previously evaluated in students with disabilities). Core implementation functions did not differ between VIT-TAY and VR-JIT. Greater engagement with either training was significantly associated with employment (both p < 0.05). Overall, VIT-TAY was feasibly implemented with high adherence and may be effective at enhancing employment for students with disabilities. Limitations and implications for research and practice are discussed.

Job Interview and Vocational Outcomes Among Transition-Age Youth Receiving Special Education Pre-Employment Transition Services.

Vocational outcomes among transition-age youth receiving special education services are critically poor and have only incrementally improved since the implementation of the Workforce Innovation Opportunity Act. Few studies highlight whether interviewing may be critical to obtaining vocational outcomes such as competitive employment or internships. This study evaluated vocational interviewing and outcomes among 656 transition-age youth receiving special education pre-employment transition services from 47 schools. Results suggest 20.8% of these youth were currently employed, and 88.8% of these employed youth interviewed prior to obtaining their job, which is higher than anecdotal evidence suggests and speaks to the importance of job interview skills as an intervention target for special education pre-employment transition services. We discuss the implications and directions for further study.

Virtual interview training for autistic transition age youth: A randomized controlled feasibility and effectiveness trial.

LAY ABSTRACT: Autistic transition age youth struggle with obtaining employment, and interviewing is a critical barrier to getting a job. We adapted an efficacious virtual reality job interview intervention to meet the needs of autistic transition age youth, called the Virtual Interview Training for Transition Age Youth. This study evaluated whether Virtual Interview Training for Transition Age Youth can be feasibly delivered in high school special education settings and whether Virtual Interview Training for Transition Age Youth improves job interview skills, job interview self-efficacy, job interview anxiety, and access to employment. Forty-eight autistic transition age youth received school-based pre-employment services as usual with Virtual Interview Training for Transition Age Youth, while 23 autistic transition age youth received services as usual only. Local teachers trained and supervised autistic transition age youth using Virtual Interview Training for Transition Age Youth. Participants reported Virtual Interview Training for Transition Age Youth was highly acceptable. Participants receiving services as usual and Virtual Interview Training for Transition Age Youth, compared to participants receiving services as usual only, had better job interview skills and lower job interview anxiety as well as greater access to jobs. Overall, Virtual Interview Training for Transition Age Youth appears to be effective at teaching job interview skills that are associated with accessing competitive jobs. Moreover, youth enjoyed Virtual Interview Training for Transition Age Youth and teachers feasibly implemented the tool within special education pre-employment transition services. Future research needs to better understand how autistic transition age youth from culturally diverse backgrounds and different social, behavioral, or mental health challenges may respond to Virtual Interview Training for Transition Age Youth.

Virtual Reality Job Interview Training in Transition Services: Results of a Single-Arm, Noncontrolled Effectiveness-Implementation Hybrid Trial.

This study reports the results of a single-arm non-controlled, Type 3 hybrid effectiveness-implementation trial evaluating virtual reality job-interview training (VR-JIT) delivered in five pre-employment transition programs comprising 15 schools, 10 administrators, 23 teachers, and 279 youth ages 16-21 years receiving special education pre-employment transition services. Fidelity, expected implementation feasibility, and teacher and student acceptance of VR-JIT were high. Youth completed virtual interviews over six to eight weeks (M = 10.8, SD = 7.4). At the six-month follow-up, teachers reported that youth using VR-JIT had employment rates higher than current national employment rates for youth with disabilities. A multinomial logistic regression revealed VR-JIT engagement was associated with greater employment rates by six-month follow-up (OR=1.63, p=0.002). This study provides promising evidence that VR-JIT may be feasibly implemented with high fidelity in special education pre-employment transition services and can potentially enhance employment outcomes among transition-age youth receiving special education services.

Using community-engaged methods to adapt virtual reality job-interview training for transition-age youth on the autism spectrum.

BACKGROUND: Virtual Reality Job-Interview Training (VR-JIT) is an efficacious Internet-based intervention for adults with severe mental illness (SMI). Evaluations of VR-JIT have shown improved interview skill and access to employment in several cohorts of adults with SMI and with autism spectrum disorders (ASD). VR-JIT trains participants how to fill out job applications and handle job interviews through e-learning content and applied practice. Trainees receive feedback through in-the-moment nonverbal cues, critiques, and recommendations for improving performance. Our study sought to adapt VR-JIT for transition-age youth with ASD (TAY-ASD). METHODS: We recruited TAY-ASD and adult stakeholders from public and charter schools, transition programs, and community service providers. Participants provided feedback on VR-JIT to enhance its applicability to TAY-ASD. We used community-engaged methods to process and analyze data from TAY-ASD and stakeholders, presented their quantitative and qualitative responses to community and scientific advisory boards for review and recommendations, and adapted the intervention design and content. RESULTS: Our adaptations included adding diversity (gender; race/ethnicity) to the virtual hiring manager; shortening the interview by reducing response options; increasing social storytelling to enhance engagement with VR-JIT core components; adding employment opportunities more relevant to younger workers; reducing the reading level; and making the e-learning content more accessible by adding bullet points, voiceover, and imagery/video; and adding new learning goals. CONCLUSIONS: This study presents a rigorous and innovative community-engaged methodology for adapting VR-JIT to meet the needs of TAY-ASD. We review our engagement with TAY-ASD and stakeholders, and discuss the standardized coding scheme we used to adapt VR-JIT and the usefulness and limitations of employing this methodology in adapting other behavioral interventions.