Gamified Assessment of the Emotion Regulation Abilities in Youth: Validation of the REThink Online System.

Objective: Emotional problems in the youth population are a major issue that can have a significant negative impact for their future development as adults. Their emotion regulation (ER) abilities represent a preventive measure for those emotional problems. REThink is an online therapeutic game that was proved to be effective in rigorous studies, and is based on the rational emotive behavioral therapy, with seven levels built to train various emotional regulation skills. Each level has a section that can be used for the assessment of the targeted skills within the level. The present study aimed at investigating the reliability and validity of the evaluation modules from the REThink game regarding the assessment of ER abilities in children/adolescents. Methods: In accordance with established guidelines, 110 children and adolescents aged 8-14 years old were recruited. Following parental informed consent, the participants filled out the standard questionnaires and, subsequently, they played the evaluation module of the REThink game. The reliability aspect was investigated by evaluating internal consistency, while validity was evaluated by using concurrent and predictive validity analyses. Results: The results revealed statistically significant positive associations between the game scores obtained by the participants and the emotion regulation scale. In terms of predictive validity, there were significant negative associations between game scores and the presence of emotional and behavioral problems. Moreover, in terms of the reliability of the REThink game, an acceptable value for the internal consistency was observed. Conclusion: In conclusion, the REThink therapeutic game was proved to be a valid measure for assessing emotion regulation abilities in children and adolescents. Clinical Trial Registration No. NCT04788901.

Predictive Validity, Applicant Reactions, and Influence of Personal Characteristics of a Gamefully Designed Assessment / [Validez predictiva, reacciones de los candidatos e influencia de las características personales de una evaluación que utiliza juegos]

The use of game-related assessments (GRAs) in personnel selection is increasing. However, the type of GRA under investigation should be considered. This study focuses on game-based assessments, which closely resemble leisure-oriented videogames. We specifically examined a serious game that assesses personality traits (assertiveness, sociability, tolerance, and rules). We analyzed its predictive validity, applicant reactions, and the impact of personal and job-related factors on scores. The study included 182 participants who played the GRA and completed a Big Five questionnaire. Findings revealed that the game predicts adaptive performance, offering incremental value over the Big Five (ΔR2 = .107). Additionally, candidates reacted more positively to the GRA in terms of comfort and fit, although the effect size was small (d = -0.1757 and -0.3119, respectively). Relating to personal and job-related characteristics, only job experience significantly affected the assertiveness score, albeit with a small effect size (d = -0.3870).(AU)

Aumenta en el campo de la selección de personal la utilización de la evaluación relacionada con juegos (ERJ). El estudio se centra en la evaluación mediante juegos, un tipo de ERJ cercano a los videojuegos recreativos convencionales. En concreto se analizó un juego serio que mide rasgos de personalidad (asertividad, sociabilidad, tolerancia y reglas), investigando su validez predictiva, las reacciones de los candidatos y la influencia de las variables personales y laborales. En la investigación participaron 182 personas que jugaron a la ERJ y cumplimentaron un cuestionario de personalidad Big Five. Los resultados muestran la validez incremental de la ERJ con respecto a los Big Five en la predicción del desempeño adaptativo (ΔR2 = .107). Además, las reacciones fueron más positivas a la comodidad y al ajuste al puesto para la ERJ que para la prueba de personalidad, aunque con un tamaño del efecto modesto (d = -0.1757 y -0.3119). Con respecto a las características personales y laborales, la experiencia laboral se asoció a una mayor asertividad, aunque el efecto también es pequeño (d = -0.3870).(AU)

[A Novel Three-minute Game-based Cognitive Risk Screening Tool-WeChat Mini-program-based Design and Large-sample Feasibility Studies].

OBJECTIVE: To develop a novel cognitive screening tool for older adults in China. METHODS: "Game-based Cognitive Assessment-3 Minute Version"(G3) was designed and developed based on WeChat mini-program. And its feasibility was analyzed. RESULTS: G3 mini-program contains three one-minute mini digital games and supports users' self-assessment of cognitive functions with instant access to reports. G3 had a good correlation with Montreal Cognitive Assessment Basic (MoCA-B) with Pearson's r =0.611 (P<0.001). Among natural users aged 50 and older (71 179), the G3 initiation and completion rates were 99.55% and 92.28%, respectively. The average time to complete G3 assessments was (278.5±73.73) seconds. CONCLUSIONS: The novel G3 mini-program has good feasibility and usability for older Chinese adults, and can be used for cognitive screening and home self-assessment.

Faster ≠ Smarter: Children with Higher Levels of Ability Take Longer to Give Incorrect Answers, Especially When the Task Matches Their Ability.

The stereotype that children who are more able solve tasks quicker than their less capable peers exists both in and outside education. The F > C phenomenon and the distance-difficulty hypothesis offer alternative explanations of the time needed to complete a task; the former by the response correctness and the latter by the relative difference between the difficulty of the task and the ability of the examinee. To test these alternative explanations, we extracted IRT-based ability estimates and task difficulties from a sample of 514 children, 53% girls, M(age) = 10.3 years; who answered 29 Piagetian balance beam tasks. We used the answer correctness and task difficulty as predictors in multilevel regression models when controlling for children's ability levels. Our results challenge the 'faster equals smarter' stereotype. We show that ability levels predict the time needed to solve a task when the task is solved incorrectly, though only with moderately and highly difficult items. Moreover, children with higher ability levels take longer to answer items incorrectly, and tasks equal to children's ability levels take more time than very easy or difficult tasks. We conclude that the relationship between ability, task difficulty, and answer correctness is complex, and warn education professionals against basing their professional judgment on students' quickness.

Compensatory beliefs in the internet gratification behavior: A study of game-based assessment.

Internet gratification behaviors (IGB) may lead to sub-optimal performance in schools and workplace as well as mental health problems such as Internet addiction. The present research examines this phenomenon, focusing on the compensatory belief (CB)-a belief that the negative impact of a certain behavior can be compensated or neutralized by another positive behavior-as a potential psychological mechanism for IGB. An interactive-narrative-style game-based assessment was designed and responses from a random-sampled population of 1,298 participants including college students and organizational employees were collected online. It was found that around 40% of college students and organizational employees would activate some kind of compensatory beliefs when facing with the internet temptation. Those who failed to perform compensatory behaviors afterwards were more likely to regret than those who were able to perform them, which was consistent with the prediction of the CB theory. This study expands the applicability of the CB theory to the field of internet addiction, enriches the understanding of the psychological mechanisms of internet addiction, and suggests that the interactive-narrative-style game-based assessment may be a practical method to study the CB.

The Development of Spatial Cognition and Its Malleability Assessed in Mass Population via a Mobile Game.

Spatial cognition is a fundamental aspect of human intelligence, but our understanding of its developmental trajectory across the life span is limited. Here, we applied game-based assessment on mobile devices to engage a large sample from China (N = 216,713) with a wide age range (from under 10 years old to above 60) in multiple participations of a mental rotation task, a typical measure of spatial cognition. We found that spatial ability developed asynchronously with its malleability. Whereas mental rotation performance peaked at the age of 28, with males performing better than females, the effect of training from repeated participation peaked at 18, probably laying the foundation for the development of spatial ability. In contrast, children showed particularly low malleability, and a follow-up experiment revealed that the underdeveloped ability of mirror-image discrimination likely hindered the malleability of spatial cognition during this period. The intermingled relation of ability and malleability illustrates dynamics in the development of spatial cognition, inviting broad research on the development of other cognitive functions.

A Novel Three-minute Game-based Cognitive Risk Screening Tool-WeChat Mini-program-based Design and Large-sample Feasibility Studies / 中国医疗器械杂志

OBJECTIVE@#To develop a novel cognitive screening tool for older adults in China.@*METHODS@#"Game-based Cognitive Assessment-3 Minute Version"(G3) was designed and developed based on WeChat mini-program. And its feasibility was analyzed.@*RESULTS@#G3 mini-program contains three one-minute mini digital games and supports users' self-assessment of cognitive functions with instant access to reports. G3 had a good correlation with Montreal Cognitive Assessment Basic (MoCA-B) with Pearson's r =0.611 (P<0.001). Among natural users aged 50 and older (71 179), the G3 initiation and completion rates were 99.55% and 92.28%, respectively. The average time to complete G3 assessments was (278.5±73.73) seconds.@*CONCLUSIONS@#The novel G3 mini-program has good feasibility and usability for older Chinese adults, and can be used for cognitive screening and home self-assessment.

Motivational traits: An objective behavioral test using a computer game.

An objective behavioral test for motivational traits has been developed taking as a reference the structure of the Motivational Trait Questionnaire, which is a validated self-report questionnaire to measure motivational traits in the population. The objective test consists of a computer game, which makes it possible to evaluate a person's motivational traits and to display them on the same scale as the Motivational Trait Questionnaire. To evaluate the validity of the results obtained in the measurement of motivational traits using the objective behavioral test, a study was carried out with 31 participants whose motivational traits were evaluated using the two methods, and the results obtained were compared to find out whether the two forms of assessment can be considered equivalent. Statistical tests support the high degree of similarity of the results, concluding that the objective behavioral test can be a useful instrument to assess motivational traits as an alternative to the self-report questionnaire. Despite the increasing popularity of game-based assessment of personality traits, this is the first time a game has been designed for the assessment of motivational traits. Being able to obtain a model of the user's motivational traits without having to rely on a questionnaire opens the possibility to build software applications that adapt to the user's motivational profile, providing them with the kind of motivational support that best fits their needs.

Assessing young children's national identity through human-computer interaction: A game-based assessment task.

As a way of human-computer interaction, game-based assessment is more suitable for young children because it is situational, interesting, and effective. National identity is an important factor affecting the overall development of young children and the future development of a country, which has attracted extensive attention from researchers. Nevertheless, the assessment of young children's national identity is still based on traditional evaluation, including questionnaires and interviews, which have the limitations of being inaccurate, dull, and time-consuming. To understand the characteristics of children's national identity, it is necessary to use scientific and interactive assessment methods. The present study investigated whether the game-based assessment we developed specifically would be an appropriate tool to measure young children's national identity. The results show that the game-based assessment had good item discrimination. Exploratory factor analysis demonstrated the game covered three aspects: national cognition mastery, national emotion engagement, and national behavior tendency. The confirmatory factor analysis suggested that the model with three factors fit the data well. The internal consistency, the split-half reliability, and the test-retest reliability meet standards. Overall, the results indicated that this game can be successfully used to assess young children's national identity with acceptable validity and reliability. Our study provides strong evidence for the use of human-computer interaction in child measurement. These findings are the first to demonstrate the promise of game-based assessment in assessing children's national identity reliably and effectively.

Game-related assessments for personnel selection: A systematic review.

Industrial development in recent decades has led to using information and communication technologies (ICT) to support personnel selection processes. One of the most notable examples is game-related assessments (GRA), supposedly as accurate as conventional tests but which generate better applicant reactions and reduce the likelihood of adverse impact and faking. However, such claims still lack scientific support. Given practitioners' increasing use of GRA, this article reviews the scientific literature on gamification applied to personnel selection to determine whether the current state of the art supports their use in professional practice and identify specific aspects on which future research should focus. Following the PRISMA model, a search was carried out in the Web of Science and Scopus databases, identifying 34 valid articles, of which 85.3% are empirical studies that analyze five areas: (1) validity; (2) applicant reactions; (3) design of GRA; (4) personal characteristics and GRA; and (5) adverse impact and faking. Together, these studies show that GRA can be used in personnel selection but that the supposed advantages of GRA over conventional tests are fewer than imagined. The results also suggest several aspects on which research should focus (e.g., construct validity, differences depending on the type of game, prediction of different job performance dimensions), which could help define the situations in which the use of GRA may be recommended.

Construct and Predictive Validity of an Assessment Game to Measure Honesty-Humility.

Research on commercial computer games has demonstrated that in-game behavior is related to the players' personality profiles. However, this potential has not yet been fully utilized for personality assessments. Hence, we developed an applied (i.e., serious) assessment game to assess the Honesty-Humility personality trait. In two studies, we demonstrate that this game adequately assesses Honesty-Humility. In Study 1 (N = 116), we demonstrate convergent validity of the assessment game with self-reported Honesty-Humility and divergent validity with the other HEXACO traits and cognitive ability. In Study 2 (N = 287), we replicate the findings from Study 1, and also demonstrate that the assessment game shows incremental validity-beyond self-reported personality-in the prediction of cheating for financial gain, but not of counterproductive work and unethical behaviors. The findings demonstrate that assessment games are promising tools for personality measurement in applied contexts.

Game based assessments of cognitive ability in recruitment: Validity, fairness and test-taking experience.

Gamification and machine learning are emergent technologies in recruitment, promising to improve the user experience and fairness of assessments. We test this by validating a game based assessment of cognitive ability with a machine learning based scoring algorithm optimised for validity and fairness. We use applied data from 11,574 assessment completions. The assessment has convergent validity (r = 0.5) and test-retest reliability (r = 0.68). It maintains fairness in a separate sample of 3,107 job applicants, showing that fairness-optimised machine learning can improve outcome parity issues with cognitive ability tests in recruitment settings. We show that there are no significant gender differences in test taking anxiety resulting from the games, and that anxiety does not directly predict game performance, supporting the notion that game based assessments help with test taking anxiety. Interactions between anxiety, gender and performance are explored. Feedback from 4,778 job applicants reveals a Net Promoter score of 58, indicating more applicants support than dislike the assessment, and that games deliver a positive applicant experience in practise. Satisfaction with the format is high, but applicants raise face validity concerns over the abstract games. We encourage the use of gamification and machine learning to improve the fairness and user experience of psychometric tests.

Reliability and validity of a computer game-based tool of upper extremity assessment for object manipulation tasks in children with cerebral palsy.

INTRODUCTION: A computer game-based upper extremity (CUE) assessment tool is developed to quantify manual dexterity of children with Cerebral Palsy (CP). The purpose of this study was to determine test-retest reliability of the CUE performance measures (success rate, movement onset time, movement error, and movement variation) and convergent validity with the Peabody Developmental Motor Scale version 2 (PDMS-2) and the Quality of Upper Extremity Skills Test (QUEST). METHODS: Thirty-five children with CP aged four to ten years were tested on two occasions two weeks apart. RESULTS: CUE performance measures of five chosen object manipulation tasks exhibited high to moderate intra-class correlation coefficient (ICC) values. There was no significant difference in the CUE performance measures between test periods. With few exceptions, there was no significant correlation between the CUE performance measures and the PDMS-2 or the QUEST test scores. CONCLUSIONS: The high to moderate ICC values and lack of systematic errors indicate that the CUE assessment tool has the ability to repeatedly record reliable performance measures of different object manipulation tasks. The lack of a correlation between the CUE and the PDMS-2 or QUEST scores indicates that performance measures of these assessment tools represent distinct attributes of manual dexterity.

Applying Learning Analytics to Detect Sequences of Actions and Common Errors in a Geometry Game.

Games have become one of the most popular activities across cultures and ages. There is ample evidence that supports the benefits of using games for learning and assessment. However, incorporating game activities as part of the curriculum in schools remains limited. Some of the barriers for broader adoption in classrooms is the lack of actionable assessment data, the fact that teachers often do not have a clear sense of how students are interacting with the game, and it is unclear if the gameplay is leading to productive learning. To address this gap, we seek to provide sequence and process mining metrics to teachers that are easily interpretable and actionable. More specifically, we build our work on top of Shadowspect, a three-dimensional geometry game that has been developed to measure geometry skills as well other cognitive and noncognitive skills. We use data from its implementation across schools in the U.S. to implement two sequence and process mining metrics in an interactive dashboard for teachers. The final objective is to facilitate that teachers can understand the sequence of actions and common errors of students using Shadowspect so they can better understand the process, make proper assessment, and conduct personalized interventions when appropriate.

Eye Tracking the Feedback Assigned to Undergraduate Students in a Digital Assessment Game.

High-quality feedback exerts a crucial influence on learning new skills and it is one of the most common psychological interventions. However, knowing how to deliver feedback effectively is challenging for educators in both traditional and online classroom environments. This study uses psychophysiological methodology to investigate attention allocation to different feedback valences (i.e., positive and negative feedback), as the eye tracker provides accurate information about individuals' locus of attention when they process feedback. We collected learning analytics via a behavioral assessment game and eye-movement measures via an eye tracker to infer undergraduate students' cognitive processing of feedback that is assigned to them after completing a task. The eye movements of n = 30 undergraduates at a university in Western Canada were tracked by the EyeLink 1000 Plus eye tracker while they played Posterlet, a digital game-based assessment. In Posterlet, students designed three posters and received critical (negative) or confirmatory (positive) feedback from virtual characters in the game after completing each poster. Analyses showed that, overall, students attended to critical feedback more than to confirmatory feedback, as measured by the time spent on feedback in total, per word, and per letter, and by the number of feedback fixations and revisits. However, there was no difference in dwell time between valences prior to any feedback revisits, suggesting that returning to read critical feedback more often than confirmatory feedback accounts for the overall dwell time difference between valences when feedback is assigned to students. The study summarizes the eye movement record on critical and confirmatory feedback, respectively. Implications of this research include enhancing our understanding of the differential temporal cognitive processing of feedback valences that may ultimately improve the delivery of feedback.

Dynamic Bayesian Network Modeling of Game-Based Diagnostic Assessments.

Digital games offer an appealing environment for assessing student proficiencies, including skills and misconceptions in a diagnostic setting. This paper proposes a dynamic Bayesian network modeling approach for observations of student performance from an educational video game. Drawing from and advancing methods in dynamic Bayesian networks, cognitive diagnostic modeling, and analysis of process data, a Bayesian approach to model construction, calibration, and use in facilitating inferences about students on the fly is described, and implemented in the context of an educational video game.