Item analysis of multiple-choice questions in anatomy for first year MBBS

Background: Multiple choice questions (MCQs) are preferred tools of assessment because of objectivity, ease of scoring and how each MCQ functions as an item can be understood by Item analysis. Aims and Objectives: The aim of the study was (i) to carry out item analysis of MCQs used in formative assessment to know the validity and (ii) to carry out a post validation item analysis of MCQ’s of 1st MBBS students in anatomy to use results for further actions. Materials and Methods: 45 MCQs were administered to 112 students of 1st M.B.B.S. as a formative assessment. Difficulty index and Discrimination index were calculated. Results: Mean difficulty index was 56.67 ± 22.09, and mean discrimination index was 0.35 ± 0.23. Distribution of easy, moderate, and difficult MCQ was 20, 67, and 13%, respectively. About 20% of MCQs were poor, 20% with acceptable discriminating index, 27% had good, and 33% MCQ were of excellent discrimination index. No item was negatively discriminating and all distractor were functional. Very easy and very difficult items had poor discriminating index. Conclusion: Most of items had moderate difficulty and good to excellent discrimination. Too easy and too difficult items showcased poor discrimination, no negatively discriminating item and absence of non-functional distractor suggest good quality framing of the MCQs.

Assessing an assessment tool: Analysis of multiple choice questions on difficulty level and discrimination power, from an assessment in physiology

Background: Multiple Choice Questions (MCQs) or ‘‘items” are frequently used in medical education to assess students’ knowledge at the end of an academic session. However, item analysis has to be done to confirm their excellence. This piece of work was aimed to analyze test items in the subject of Physiology. In view of the fact that item analysis has never been conducted in our physiology department, thus this study was planned to assess the quality of MCQs and construct a viable question bank. Aim and Objectives: To evaluate the quality of MCQs used so as to develop a pool of valid items to update the question bank. Materials and Methods: Total of one hundred 1st year MBBS students were given 40 MCQs. Analysis was done with the item difficulty and item discrimination indices. Results: Difficulty indices (p) of 14 items were easy while about two MCQ were difficult and the remaining 24 of the items were within acceptable range. The discrimination index (DI) (D) for 40 items; one item showed negative discrimination, ten were having poor discrimination, whereas 11 were having excellent discrimination.Nine items were having acceptable, and nine items were having good discrimination. The DI exhibited slight positive correlation with difficulty index (r = 0.1575 P = 0.05). However, it was not statistically significant. Conclusion: The findings show that most of the items were falling in acceptable range of difficulty and discrimination level. It concludes that Item analysis is a very useful and necessary procedure to obtain valid Items from time to time.

Role of Multiple Choice Questions in Assessment of Competency of Knowledge in Anatomy

Objectives: The main objective of the current study is to examine the items using the item analysis to assess their competency of knowledge in human anatomy. Methods: This is a quantitative descriptive cross-sectional study conducted at the Najran University faculty of Medicine, in the department of anatomy. A 31-second year MBBS students of level three took the multiple-choice question exam comprising 100 questions in one session. Each item was analyzed by item analysis. The planned period of data collection is the first semester of the academic year 2019-2020. The data were analyzed using SPSS version 25, and Excel Software. All analyses were performed using the descriptive frequency, explore and correlation probabilities a p-value, p<0.01 (2-sided) was considered to indicate statistical significance. Results: The assessment of test items use the item analysis that is very difficult question: 25; 25%, difficult question: 8; 8%, average question: 46; 46%, easy questions; 9; 9% and very easy: 9; 9%. The discrimination indices, which are poor discrimination; 34; 34%, acceptable discrimination; 11; 11%,good discrimination; 6; 6%, very good discrimination; 4; 4% and excellent discrimination; 45; 45%. The performance of students in this test showing that a large number of items have a role in distinguishing students knowledge in the anatomy test, but in Simple parts of the scale that the items did not succeed in assessing students. Items of anatomy exams displayed a strong relationship of difficulty (P) and discrimination (D) indices. Conclusion: The MCQs items assess their competency of knowledge in human anatomy. The competence of students in the knowledge of anatomy measured at a high rate of A-type context-rich MCQs can be stem is case scenario, lead-in; question and options; distractors, and key. These items can differentiate good and poor accomplishment students..

Item analysis as tool to validate multiple choice question bank in pharmacology

Background: Multiple choice questions (MCQs) are a common method for formative and summative assessment of medical students. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), distracter efficiency (DE). The objective of this study was to assess the quality of MCQs currently in use in pharmacology by item analysis and develop a MCQ bank with quality items.Methods: This cross-sectional study was conducted in 148 second year MBBS students at NKP Salve institute of medical sciences from January 2018 to August 2018. Forty MCQs twenty each from the two term examination of pharmacology were taken for item analysis A correct response to an item was awarded one mark and each incorrect response was awarded zero. Each item was analyzed using Microsoft excel sheet for three parameters such as DIF I, DI, and DE.Results: In present study mean and standard deviation (SD) for Difficulty index (%) Discrimination index (%) and Distractor efficiency (%) were 64.54±19.63, 0.26±0.16 and 66.54±34.59 respectively. Out of 40 items large number of MCQs has acceptable level of DIF (70%) and good in discriminating higher and lower ability students DI (77.5%). Distractor efficiency related to presence of zero or 1 non-functional distrator (NFD) is 80%.Conclusions: The study showed that item analysis is a valid tool to identify quality items which regularly incorporated can help to develop a very useful, valid and a reliable question bank.

Impact of distractors in item analysis of multiple choice questions

Background: Item analysis is a quality assurance of examining the performance of the individual test items that measures the validity and reliability of exams. This study was performed to evaluate the quality of the test items with respect to their performance on difficulty index (DFI), Discriminatory index (DI) and assessment of functional and non-functional distractors (FD and NFD).Methods: This study was performed on the summative examination undertaken by 113 students. The analyses include 120 one best answers (OBAs) and 360 distractors.Results: Out of the 360 distractors, 85 distractors were chosen by less than 5% with the distractor efficiency of 23.6%. About 47 (13%) items had no NFDs while 51 (14%), 30 (8.3%), and 4 (1.1%) items contained 1, 2, and 3 NFDs respectively. Majority of the items showed excellent difficulty index (50.4%, n=42) and fair discrimination (37%, n=33). The questions with excellent difficulty index and discriminatory index showed statistical significance with 1NFD and 2 NFD (p=0.03).Conclusions: The post evaluation of item performance in any exam in one of the quality assurance method of identifying the best performing item for quality question bank. The distractor efficiency gives information on the overall quality of item.

Item analysis to identify quality multiple choice questions/items in an assessment in Pharmacology of II MBBS students in Guntur Medical College of Andhra Pradesh, India

Background: In medical education, multiple choice questions/Items are the more frequently used assessment tools to assess the knowledge abilities and skills of medical students, for being their objectivity, wide coverage in less time. However only the Quality Items gives a valid and reliable assessment. The quality of an Item is determined by difficulty index (DIF I), Discrimination Index (DI) and Distractor efficiency (DE). Aim of the study was to know the quality of Items in pharmacology by Item analysis and to develop a MCQs bank with quality Items.Methods: The present study was conducted on 150 II MBBS students of Guntur Medical College, AP, India. A class test containing 50 Items with 150 distractors from topic chemotherapy was conducted. Item with the correct choice/response was awarded with one mark and with the wrong choice zero marks, no negative marks. Each test Item was analysed with DIF I, DI and DE and the results were tabulated and tested statistically, with unpaired "t" test.Results: Mean DIF I, DI, DE values with standard deviations in the present study are 44.72+17.63%, 0.30+0.12%, 84.48+24.65 respectively. DIF I of 32 (64%) items was good to excellent range (31%-60%) 9 (18%) Items were easy (>61%) and 9(18%) Items were difficult (>30%). DI of 10 (20%) Items was good (0.15 to 0.24.) 29 (58%) Items were excellent with DI > 0.25 and 11 (22%) Items were poor with DI <0.15. Among 150 distractors, 127 (85%) were functional distractors (FDs) and 23 (15%) were non-functional distractors (NFDs). DE of 33 (66%) items with nil NFDs was 100%, for 12 (24%) Items with one NFD, was 66.6%, for 4 (8%) items with 2 NFDs was 33.3% and for 1 (2%) Item with 3NFDs DE was 0%. When unpaired "t" test was applied to the means of "difficult" and "easy" Items, 96.22+11.33% SD, 51.44+29.31% SD respectively, the p-value obtained was 0.00058, which was highly significant.Conclusions: The study showed that Item analysis is a valid tool to identify quality Items, which assess, the students� knowledge abilities and discriminate different levels of performance abilities of students effectively.

Multiple Choice Questions - Role In Assessment Of Competency Of Knowledge In Anatomy

Background: The accurate, reliable and timely assessment of students is an essential domain of teaching duringMedical professional courses. The Multiple Choice Questions (MCQ) are time tested method of ready assessmentof undergraduate students. Although it evaluates student’s cognitive knowledge but does not evaluate professionalskills. However it is said that MCQs emphasize recall of factual information rather than conceptual understandingand interpretation of concepts.Objectives: The main objective of the study is to analyse the items with the help of item analysis and select theitems which are good for incorporation into future question bank with reliability.Materials and Methods: This study was done in Department of Anatomy, AIIMS, Patna. A 396 first year MBBSstudents of different batches took the MCQ test comprising 60 questions in two sessions. During the evaluationprocess of MCQ’s each correct response was awarded one mark and no marks was awarded for any incorrectresponse. Each item was analysed for difficulty index, discrimination index and distractor effectiveness.Results: The overall mean of Facilitative value, Discrimination Index, Distractor Effectiveness and CorrelationCoefficient was 66.09 (±21.55), 0.26 (±0.16), 18.84 (±10.45) and 0.55±0.22 respectively.Conclusion: The framing of MCQ should be according to Bloom’s classification to assess cognitive, affective aswell as psychomotor domain of the students. The MCQ having poor and negative discrimination should bereframed and again should be analysed.

Psychometric properties for the brazilian version of the new ecological paradigm: revised / Propriedades psicométricas da versão brasileira da escala de novo paradigma ecológico: revisada / Propiedades psicométricas de la versión brasileña de la escala nuevo paradigma ecológico: revisado

The New Ecological Paradigm (NEP-R) is one of the hallmarks for the measurement of ecological attitudes. Attention has been given to its psychometric properties during the last ten years, but no analyses have been conducted until now using an Item Response Theory (IRT) approach. With this purpose, a sample of 410 recruited responders answered the NEP-R's 15 items in Brazilian Portuguese. An item analysis was conducted considering: dimensionality; reliability; discrimination; item information; and item/person hierarchy. The results for the principal component analysis of standardized residues supports a unidimensional structure. Results for item fit coefficients (INFIT and OUTFIT) indicate that the items have good predictive power for the response patterns in the latent trait continuum representing ecocentrism. In turn, the results for person discrimination and item hierarchy point out that the scale is restricted as to the levels of difficulty it represents. The findings include that the most revealing items for environmental attitudes in a Brazilian sample consist of the set of items representing the "limits of growth" facet.

O Novo Paradigma Ecológico (NPE-R) é um dos marcos na medida de atitudes ecológicas. Suas propriedades psicométricas têm sido investigadas durante os últimos dez anos, mas não há na literatura análises disponíveis utilizando a Teoria de Resposta ao Item (TRI). Com esse objetivo, foram recrutados 410 participantes para responder os 15 itens da NPE-R em português brasileiro. Uma análise de itens foi conduzida considerando: dimensionalidade, confiabilidade, discriminação, informação do item e hierarquia de itens/pessoas. A análise de componentes principais dos resíduos padronizados suporta uma estrutura unidimensional. Resultados do ajuste dos itens pelos coeficientes de INFIT e OUTFIT indicam que os itens possuem bom poder preditivo para os padrões de resposta no continuum de traço latente representativo do ecocentrismo. Por sua vez, os resultados para o poder de discriminação da escala e a hierarquia de itens apontam que a escala é restrita quanto aos níveis de dificuldade que abarca. Os achados incluem que os itens mais reveladores de atitudes ambientais para a amostra brasileira consistem no conjunto que representa a faceta de "limites de crescimento".

El Nuevo Paradigma Ecológico (NPE-R) es uno de los marcos en la medida de las actitudes ecológicas. Sus propiedades psicométricas han sido investigadas en los últimos diez años, pero no hay un análisis de la literatura disponible usando la Teoría de Respuesta al Ítem (TRI). Para esto, 410 participantes fueron reclutados para responder los 15 ítems de la NPE-R en portugués de Brasil. Se realizó un análisis de ítems considerando: dimensionalidade, fiabilidad, la discriminación, la información de los ítems y de la jerarquía de los ítems/personas. El análisis de componentes principales de los residuos estandarizados soporta una estructura unidimensional. Los resultados del ajuste de los ítems por los coeficientes de INFIT y OUTFIT indican que los ítems tienen un buen poder predictivo de los patrones de respuesta en el continuum latent representativo del ecocentrismo. A su vez, los resultados para el poder de la discriminación escala y jerarquía de ítems señalan que la escala está restringida en cuanto a los niveles de dificultad. Los hallazgos incluyen que los elementos más reveladores de las actitudes ambientales para la muestra brasileña son formados por el conjunto de ítems de la faceta de "límites del crecimiento".

Relationship Between Difficulty Index and Distracter Effectiveness in Best-Answer Stem Type Multiple Choice Question.

Background: Single best-answer multiple-choice questions (MCQs) consist of a question (the stem) two or more choices from which examinees must choose the correct option (the distracters) and one correct or best response (the key). Item analysis is the process of collecting, summarizing and using information from students’ responses to assess the quality of test items. Classical test theory for item analysis is most followed method to determine the reliability by calculating Difficulty Index (P score) and Discriminating Index (D score) and Distracter effectiveness Aim: This Study was aimed to calculate P scoreand distracter effectiveness; to find out relationship between P score and distracter effectiveness. Material and methods: In this Cross Sectional study 65 items responded by 120 Students of first year M.B.B.S were studied for Item Analysis. Difficulty Index, and Distracter Effectiveness were calculated for each item. Distracters were identified and classified as Functioning and Non- functioning distracter. Interrelationship between P Score, and Distracter Effectiveness was calculated and analyzed by Epinifo 7 software Result: We found Items with two functioning distracters were more difficult than that of others followed by items with three functioning distracters. Conclusion: Distractors affect the item difficulty index and by the means also affects quality of the assessment .

Evaluación de conocimientos con exámenes de selección múltiple: ¿tres o cuatro opciones de respuesta? Experiencia con el examen de admisión a posgrados médico-quirúrgicos en la Universidad de Antioquia / Evaluation of knowledge with multiple-choice tests: three of four options? Experience with admission examinations to medical and surgical postgraduate studies at University of Antioquia (Medellín, Colombia) / Avaliação de conhecimentos com exames de seleção múltipla: três ou quatro opções de resposta? Experiência com o exame de admissão a pósgraduações médico-cirúrgicos na Universidade de Antioquia

Introducción: el objetivo de este estudio fue evaluar el efecto de la reducción del número de opciones de respuesta por pregunta sobre los indicadores sicométricos de un examen de ingreso a estudios médicos de posgrado. Metodología: aplicación de índices de evaluación sicométrica desde la perspectiva de dos teorías: la clásica de la medición y la de respuesta al ítem, a una prueba de 70 preguntas hecha a 2.539 aspirantes a ingresar a los posgrados médico-quirúrgicos de la Universidad de Antioquia en el año 2014. Se eliminó la opción de respuesta elegida con menor frecuencia y se la reemplazó por azar de entre las tres restantes. Resultados: solo 52,9% de las preguntas tuvieron tres opciones funcionales de respuesta. No se encontró diferencia en la dificultad, la discriminación, el error estándar de la medición, el alfa de Cronbach ni el coeficiente de correlación biserial (teoría clásica de la medición); tampoco en la medida de dificultad de los ítems o de habilidad de las personas (teoría de respuesta al ítem) entre las pruebas con tres y cuatro opciones de respuesta. La prueba con tres opciones conservó un buen ajuste. Conclusión: una prueba con tres opciones de respuesta se comportó tan bien como su contraparte de cuatro opciones.

Introduction: The aim of this study was to evaluate the effect of reducing the number of response options per question on the psychometric indicators of an exam for admission to postgraduate medical studies, at University of Antioquia, in Medellín, Colombia. Methodology: Application of psychometric assessment indexes from the perspective or two theories: the classical of measurement and the item response, to a test of 70 questions, applied in 2014 to 2.539 candidates. The least frequently chosen distractor was eliminated and randomly replaced by one of the three remaining ones. Results: Only 52.9% of the questions had three functional distractors. No difference was found in the difficulty, discrimination, standard error of measurement, Cronbach's alpha and the coefficient of biserial correlation (classical measurement theory). Also, there was no difference in the extent of item difficulty or ability of people (item response theory). The test with three options retained a good fit. Conclusion: Multiple choice tests with three response options performed as well as their four options counterparts.

Introdução: o objetivo deste estudo foi avaliar o efeito da redução do número de opções de resposta por pergunta sobre os indicadores psicométricos de um exame de rendimento a estudos médicos de pós-graduação. Metodologia: aplicação de índices de avaliação psicométrica desde a perspectiva de duas teorias: a clássica da medição e a de resposta ao item, a uma prova de 70 perguntas feita a 2.539 aspirantes a ingressar às pós-graduações médico-cirúrgicos da Universidade de Antioquia no ano de 2014. Eliminou-se a opção de resposta eleita com menor frequência e se a substituiu por casualidade de entre as três restantes. Resultados: só 52,9% das perguntas tiveram três opções funcionais de resposta. Não se encontrou diferença na dificuldade, a discriminação, o erro regular da medição, o alfa de Cronbach nem o coeficiente de correlação bisserial (teoria clássica da medição); também não na medida de dificuldade dos itens ou de habilidade das pessoas (teoria de resposta ao item) entre as provas com três e quatro opções de resposta. A prova com três opções conservou um bom ajuste. Conclusão: uma prova com três opções de resposta se comportou tão bem como sua contraparte de quatro opções.

Using item analysis on essay types questions given in summative examination of medical college students: facility value, discrimination index.

Background: The most common tool used for assessment of knowledge is the essay questions. Their evaluations depend upon test and item analysis which is consisting of analysis of individual questions and analysis of the whole test. The objective of our study was to calculate the Item analysis (Facility value, FV and discrimination index, DI) of questions given in terminal examination of MBBS students and observes the adequacy of questions framing in examination. Methods: Study contains 150 medical students undergone terminal examination consist of questions of essay type, structured essay type and short answer questions were given to the students. We divide them into high ability group and low ability group. Mark range was decided for essay type, structured essay type and short answer type questions and FV & DI was calculated. Results: Studies described that facility values of 62.5 percentage questions were come under recommended & acceptable range and 50 percentage questions were come under acceptable range. Discrimination value of 100 percentage questions were come under recommended & acceptable range where 75 percentage questions were come under acceptable range. Conclusion: The importance of item analysis is highlighted from our results. For improvement of examination items with average difficulty and high discrimination should be implemented into future examinations to improve test scores and properly discriminate among the students.

Novos estudos psicométricos para o subteste de leiturado teste de desempenho escolar / New psychometric studies for the reading subtest of the test of school performance / Nuevos estudios psicométricos para el subtest de lectura de la prueba de rendimiento escolar

O estudo relata a análise dos itens do subteste de leitura do Teste de Desempenho Escolar (Stein, 1994), aplicado em estudantes do 2º ao 5º ano de escolarização da cidade de Belo Horizonte (N = 341). Tomando dois critérios de correção - um apresentado no manual (A1) e outro que considerou incorretas respostas de silabação e de correção espontânea (A2) - foi selecionado um conjunto de itens para cada um destes. Apenas 28,6% dos itens originais apresentaram índices adequados de discriminação pelo Critério A1 e 48,6% pelo A2. A proporção de acertos para ambos os critérios foi elevada. Normas intragrupo aos novos conjuntos de itens foram fornecidas. Os resultados indicam que o Critério A2 produziu resultados mais promissores do que o A1.

This study reports the item analysis of the reading subscale of the School Performance Test (Stein, 1994) answered by 341 children from 2nd to 5th grades of Belo Horizonte city. Two sets of items were selected taking as reference different criteria of correction - one presented in the test manual (A1) and another, which considered syllabication and self-correction as incorrect responses. Only 28.6% of the original items presented suitable discrimination indices by the A1 Criterion and 48.6% by the A2. The accuracy for both criteria was high. For each new set of items, intra-group norms were offered. The data indicate that the A2 Criterion produced more promising results than that of A1.

El estudio reporta la análisis de los ítems de la subprueba de lectura de la Prueba de Rendimiento Escolar (Stein, 1994), que se aplicó a estudiantes de 2º a 5º año de la Red de Enseñanza de Belo Horizonte (N = 341). Tomando dos criterios de corrección (uno que se presenta en el manual (A1) y otro (A2) que consideró las respuestas de silabación y de corrección espontánea cómo incorrectas) se ha seleccionado un conjunto de ítems para cada criterio. Sólo el 28,6% de los ítems originales han presentado niveles adecuados de discriminación por el criterio A1, y 48,6% por el A2. La proporción de respuestas correctas para ambos criterios se mostró alta. Se proporcionaron normas intragrupo para los nuevos conjuntos de ítems. Los resultados indican que el Criterio A2 produce resultados más prometedores que A1.

Goodness of Measurement: Reliability and Validity.

The two most important and fundamental characteristics of any measurement procedure are reliability and validity and lie at the heart of competent and effective study. However, these phenomena have often been somewhat misunderstood or under emphasized. How productive can be any research, if the instrument used does not actually measure what it purports to? How justifiable the research that is based on an inconsistent instrument? What constitutes a valid instrument? What are the implications of proper and improper testing? This paper attempts to explore these measurement related concepts as well as some of the issues pertaining thereto.

Correlation between difficulty & discrimination indices of MCQs in formative exam in Physiology.

Introduction: Difficulty index (P) and discrimination index (D) are the parameters used to evaluate the standard of multiple choice questions (MCQs) in examination. Accordingly the standard of MCQs can be interpreted as excellent, acceptable or poor. This study was intended to find out the standard of MCQs in formative examination in Physiology. The study also intended to find out correlation between P and D. Materials and Methods: There were 240 MCQ items, taken from the past 4 year batches of 100 students and were analyzed for level of difficulty and discrimination index. The relationship between them for each test item was determined by Pearson correlation analysis using SPSS 11.5. Results: There was a wide distribution of item difficulty indices (8.57 to 95.71) and discrimination indices (-0.54 to 0.8).The mean difficulty index (P) was 52.53 + 20.59 and mean discrimination index was 0.30+ 0.18. On average, about 23% of the MCQ items were easy (P >70%), while about 15% were difficult (P <30%). The remaining 62% items were within acceptable range (30 to 70%). In all 4% of the items showed negative discrimination and 21% of the items exhibited poor discrimination. The remaining 75% of the items were in the range of acceptable to excellent discrimination. The discrimination index exhibited slight positive correlation with difficulty index (r = 0.191, P=0.003<0.01). The maximal discrimination (D=0.6-0.8) was observed with moderately easy/difficult items (P = 40% - 60%). Conclusion: The majority (75%) of the items was acceptable as far as difficulty and discriminative indices were concerned. Moderately easy/difficult items had maximal discriminative ability. Too easy and too difficult items gave poor discrimination index. Negative discrimination was observed in only 4% of the items indicating faulty items or incorrect keys.

Strategic use of MCQs in undergraduate medical students to improve objectivity of formative assessment.

Background and Objectives: Assessment is a matter of continuing concern for medical teacher as it is supposed to steer student learning. Globally there is an increasing trend to move from subjectivity to objectivity. While the universities are custodial for modifying the examinations system as a summative assessment, what is more important for teachers is to mould their formative assessment. University examinations are criticized to be highly subjective and inadequate for covering full range of topics in Pharmacology. Further there is little scope for providing feedback to the students for continuous learning and improvement. In response to this it was decided to introduce multiple choice questions (MCQs) as a tool for formative assessment. Methodology: In the present study, the multiple choice questions were given along with the subjective questionsfrom the same topic to the students. The main aim was to improve the objectivity in students’ assessment by introducing MCQs as a formative assessment tool. To evaluate the affectivity of this pattern of assessment from the student’s viewpoint, we designed a feed back questionnaire which was analyzed. The MCQs were evaluated by doing Item analysis. Results: The overall feedback revealed that 89% of the students were of opinion that MCQs preparation and examination helped them in learning the subject, self studying, reasoning and enhancement of clinical skills. The present study also clearly indicates that MCQs alone are not sufficient to assess learning and should be used along with subjective tests. Conclusion: To make testing and assessment both fair and consequently valid, MCQs should be used strategically to test important subject content and it should be aligned with subjective tests.

Construção de uma tarefa de leitura em voz alta de palavras: análise psicométrica dos itens / The construction of a word reading aloud task: a psychometric item analysis report

O estudo descreve a análise dos itens que irão compor uma tarefa de leitura em voz alta de palavras isoladas. Um grupo de crianças do 2º ao 5º ano do Ensino Fundamental leu, por meio de microcomputadores, uma lista contendo 323 palavras de baixa frequência. Foram selecionados 112 itens-candidatos que apresentaram índices adequados de dificuldade, discriminação, correlação item-total e consistência interna. A análise mostrou que a maioria das palavras apresentou níveis de dificuldade baixos, mas paralelamente produziu níveis adequados de discriminação e de correlação item-total. Elevados índices de consistência interna foram obtidos, indicando a homogeneidade da tarefa e a unidimensionalidade do construto avaliado. Considerações de Embretson (1983) sobre as pesquisas de representação do construto e de intervalo nomotético são discutidas...

The study reports the psychometric inquiry of a word reading aloud task. A group of 1st to 4th grade school children read aloud 323 low frequency words that were presented in a computer screen. The item analysis selected 112 potential items that presented satisfactory levels of difficulty, discrimination, item total correlation, and internal consistency. Generally, the selected items were not difficult to the sample. Nevertheless, most items had good levels of discrimination and item total correlation. The internal consistency index attested for the homogeneity of the task and for the construct unidimensionality. Considerations of Embretson (1983) about construct representation research and nomothetic span research are outlined...

Research on Development of "Psychological Stress Questionnaire of China Civil Aviation Pilots" / 航天医学与医学工程

Objective To develop "Psychological Stress Questionnaire of China Civil Aviation Pilots(PSQ-CCAP)"according to their vocational characteristics.Methods Firstly based on literature reviews,expert interviews and open questionnaire investigations,the preliminary psychological stress questionnaire,with 47 items,was developed.Preliminary test data(N1=144) were processed with factor analysis,reliability and validity detection.Secondly 180 questionnaires were sent to various airline civil aviation pilots,165 were back.Formal test data(N2=315) were checked with factor analysis.Results "PSQ-CCAP" consisted of work load,colleague relation,career development,requirements of aviation safety,work environment and life event stresses,6 dimensions,total with 32 items.Good reliability and validity(with c coefficient 0.706~0.897) met the requirements of PSQ-CCAP.Conclusion "PSQ-CCAP" is developed.It can be applied to psychological stress measurement of China civil aviation pilots and further related researches.

Experience of Objective Structured Clinical Examination in Gyeong-Sang National University College of Medicine / 한국의학교육

PURPOSE: The third year students at Gyeong-Sang National University College of Medicine were asked to perform objective structured clinical examination (OSCE) at the end of the 2004 academic year. In this article, we analysed the correlation among the scores of OSCE with those of Final Term Examination in Internal Medicine (FTE), Clerkship Performance in Internal Medicine (CP) and Case Conference Examination (CCE), which assess knowledge, skill and attitude, and problem solving ability in medicine, respectively. We also analyzed difficulty and discrimination indexes of the test questions or evaluation criteria, and assessed their objectivity. METHODS: 1) Relevance: the pearson correlation analysis was performed on the scores of 85 students from 6 OSCE stations, FTE, CP and CCE. 2) Analysis on the evaluation criteria: the difficulty and discrimination indexes of the 59 evaluation criteria were calculated. 3) Objectivity: in order to assess objectivity in the scores, the Student t-test was performed on the scores of students from 4 OSCE stations where there was a change in the examiners after the morning sessions and on the students from 2 OSCE stations that had no change of examiners. RESULTS: 1) Relevance: Correlation coefficients between the scores of OSCE and FTE, CP or CCE were 0.335, 0.326 and 0.421, respectively (p < 0.01). 2) Analysis on the evaluation criteria: difficulty indexes of the 59 criteria ranged from 0.15 to 0.98, and 43% of the criteria belonged to the acceptable range. Discrimination indexes ranged from -0.22 to 0.61, and 69% of the criteria were acceptable. 3) Objectivity: The student t-test showed significant differences between the scores of the morning and afternoon sessions in 2 of the 4 stations, where the examiners were changed in the afternoon (p < 0.01). CONCLUSION: Our results suggest that OSCE performed in 2004 may have relevance to a valid tool in assessing clinical competence of medical students in terms of knowledge, skill, attitude and problem solving ability. However, in order to reuse the test materials in the future, evaluation criteria need to be refined further based on difficulty and discrimination indexes, and measures to improve objectivity of examiners should also be studied.

Software Design for Paper Quality Analysis / 中华医学教育探索杂志

Paper analysis plays an important role in teaching improvement as well as in its quality enhancement.We have developed a set of software for analyzing paper qualities,which will make paper analysis efficient,practical and standardizing.

Several Parameters and Their Usages in Software System for Paper Quality Analysis / 中华医学教育探索杂志

Degree of Confidence(?),Validity(E),Item Difficulty(P) and Item Discrimination(D) used in Software System for Paper Quality Analysis play an important role in evaluating the qualities of items and papers.Their usages and meanings are discussed in detail here,and an example is given to illustrate their significance and important roles in items analysis and papers analysis.