Peer-facilitated virtual action learning: reflecting on critical incidents during a pediatric clerkship.

OBJECTIVES: Action learning (AL) facilitates reflection, critical thinking, and learning while solving real-world problems. Virtual AL is an asynchronous collaborative process that encourages students to analyze their critical incidents and learn from these experiences. The researchers sought to: 1) determine whether medical students engaged in peer-facilitated virtual action learning (VAL) demonstrated reflection and critical thinking around complex issues during their pediatric clerkship; and 2) identify challenges students face during their clerkships. METHODS: Seventy clerkship students were introduced to reflection and participated in VAL by using an electronic discussion board. Each posted 1 critical incident and group members responded with thought-provoking questions and comments to facilitate reflection and analysis. Weekly, students who posted incidents revisited their incidents, pondered the questions posed, and wrote essays summarizing their reflections and insights gained. Data were analyzed using qualitative methods. RESULTS: Seventy incidents, 210 responses, and 70 revised incidents/essays were analyzed. Outcomes included broadened perspectives (44/70), questioned assumptions (12/70), and reconfirmed thinking (14/70). Content themes included communication, role identification, medical treatment concerns, and limited voice and power. CONCLUSIONS: Most of the students engaged in VAL demonstrated reflection on complex clinical issues. Themes portrayed struggles encountered and exposed issues in the hidden curriculum, suggesting a lack of voice and power that may lead to missed learning opportunities. Discussion threads offered the clerkship director insights into the overall student experience to improve future clerkship experiences.

Assessing reflective writing on a pediatric clerkship by using a modified Bloom's Taxonomy.

OBJECTIVE: Reflection enables learners to analyze their experiences and capture the wisdom that lies within. Effective teaching requires reliable methods of assessment. Several methods of assessing reflective writing have been described; however, they often require significant training, and reliability has seldom been assessed. This study was designed to determine the interrater reliability of a method of assessing reflective writing by using a modified Bloom's Taxonomy. METHODS: Twenty-one third-year medical students maintained reflective journals throughout their pediatric clerkship. A coding schema based on Bloom's Taxonomy was developed to assess the level of cognitive processing evident in the journals. Journals were independently assessed by 3 raters. Percent agreement, kappa statistics, and intraclass correlation coefficients (ICC [2,1]) were used to assess interrater reliability. RESULTS: Three hundred eight entries from 21 journals were assessed. Percent agreement ranged from 78.2% to 100%. Kappa statistic for each level ranged from 0.57 +/- 0.04 to 0.73 +/- 0.04, and for the highest level of processing evident it ranged from 0.52 +/- .04 to 0.58 +/- 0.04. ICC (2,1) for each level of cognitive processing ranged from 0.62 (F = 6.20; P = .000) to 1.00, and for the highest level of cognitive processing evident, it was 0.79 (F = 12.42; P = .000). Substantial to almost perfect agreement was attained. CONCLUSIONS: Reflective journals allow learners to revisit their experiences for critical analysis and deeper learning. This study describes a reliable method, based on Bloom's Taxonomy, of determining whether learners have achieved higher order thinking through reflective journal writing. This method can provide a baseline for facilitating higher order processing, critical thinking, and reflective practice.

A method for assessing reflective journal writing.

Reflection is widely accepted as a learning tool and is considered integral to professional practice. Journal writing is advocated in facilitating reflection, yet little is written about how to assess reflection in journals. The purpose of this study was to develop and test a method of assessing the elements of reflection in journals and to determine whether, and to what level, reflection occurs in journals. Twenty-seven physical therapy students maintained written reflective journals throughout three of their four eight-week clinical affiliations. The students were introduced to concepts of reflective practice with definitions of terms and reflective questions before their second affiliation. A coding schema was developed to assess the journals. Three raters assessed forty-three journals. The text of each journal was analyzed for evidence of nine elements of reflection, and each journal was categorized as showing no evidence of reflection, evidence of reflection, or evidence of critical reflection. Descriptive statistics were used to demonstrate evidence of reflection. Reliability between each pair of raters was assessed using percent agreement, phi coefficients, and gamma statistics. Interrater reliability of all raters was assessed using intraclass correlation coefficients (ICC[2,1]). Results showed that the raters assessed 95.3%-100% of the journals as showing at least one element of reflection. The percent agreement between rater pairs for the nine elements of reflection ranged from 65.1% to 93.0%, the phi coefficient ranged from 0.08 to 0.81, and the ICC(2,1) values used to assess reliability among the three raters on each element ranged from 0.03 to 0.72. Averaging the assessment of the three raters for the overall journal, 14.7% of the journals were assessed as showing no evidence of reflection, 43.4% as showing evidence of reflection, and 41.9% as showing evidence of critical reflection. The percent agreement between rater pairs for the overall assessment of the journals ranged from 67.4% to 85.7%, the gamma statistic ranged from 0.88 to 0.98, and the ICC(2,1) among all raters was 0.74 (95% confidence interval, 0.61-0.84). These results represent an acceptable level of agreement for use of this method of assessment for educational purposes. The coding schema developed provides a mechanism to assess evidence of reflection in written journals, which will enable instructors to evaluate student competency, obtain a baseline for facilitating reflective practice, and assess their own efficacy in facilitating reflection among students.