Validating Parallel-Forms Tests for Assessing Anesthesia Resident Knowledge.

We created a serious game to teach first year anesthesiology (CA-1) residents to perform general anesthesia for cesarean delivery. We aimed to investigate resident knowledge gains after playing the game and having received one of 2 modalities of debriefing. We report on the development and validation of scores from parallel test forms for criterion-referenced interpretations of resident knowledge. The test forms were intended for use as pre- and posttests for the experiment. Validation of instruments measuring the study's primary outcome was considered essential for adding rigor to the planned experiment, to be able to trust the study's results. Parallel, multiple-choice test forms development steps included: (1) assessment purpose and population specification; (2) content domain specification and writing/selection of items; (3) content validation by experts of paired items by topic and cognitive level; and (4) empirical validation of scores from the parallel test forms using Classical Test Theory (CTT) techniques. Field testing involved online administration of 52 shuffled items from both test forms to 24 CA-1's, 21 second-year anesthesiology (CA-2) residents, 2 fellows, 1 attending anesthesiologist, and 1 of unknown rank at 3 US institutions. Items from each form yielded near-normal score distributions, with similar medians, ranges, and standard deviations. Evaluations of CTT item difficulty (item p values) and discrimination (D) indices indicated that most items met assumptions of criterion-referenced test design, separating experienced from novice residents. Experienced residents performed better on overall domain scores than novices (P < .05). Kuder-Richardson Formula 20 (KR-20) reliability estimates of both test forms were above the acceptability cut of .70, and parallel forms reliability estimate was high at .86, indicating results were consistent with theoretical expectations. Total scores of parallel test forms demonstrated item-level validity, strong internal consistency and parallel forms reliability, suggesting sufficient robustness for knowledge outcomes assessments of CA-1 residents.

Forced choice test score of p ≤ .20 and failures on ≥ six performance validity tests results in similar Overall Test Battery Means.

Objective: To determine if similar levels of performance on the Overall Test Battery Mean (OTBM) occur at different forced choice test (FCT) p-value score failures. Second, to determine the OTBM levels that are associated with failures at above chance on various performance validity (PVT) tests. Method: OTBMs were computed from archival data obtained from four practices. We calculated each examinee's Estimated Premorbid Global Ability (EPGA) and OTBM. The sample size was 5,103 examinees with 282 (5.5%) of these scoring below chance at p ≤ .20 on at least one FCT. Results: The OTBM associated with a failure at p ≤ .20 was equivalent to the OTBM that was associated with failing 6 or more PVTs at above-chance cutoffs. The mean OTBMs relative to increasingly strict FCT p cutoffs were similar (T scores in the 30s). As expected, there was an inverse relationship between the number of PVTs failed and examinees' OTBMs. Conclusions: The data support the use of p ≤ .20 as the probability level for testing the significance of below chance performance on FCTs. The OTBM can be used to index the influence of invalid performance on outcomes, especially when an examinee scores below chance.

Coconut fatty acids exhibit strong repellency and week-long efficacy against several urban pest arthropods of the southwestern United States.

BACKGROUND: The southwestern United States is home to a variety of arthropods including Turkestan cockroaches, Blatta lateralis (Walker); hematophagous kissing bugs, Tritoma rubida (Uhler); and Arizona bark scorpions, Centruroides sculpturatus Ewing. These arthropods cause medical concern when they become established around homes and/or invade indoors. Traditionally, the management of these pests has relied primarily on the use of chemical insecticides; however, they offer poor prospects for control owing to their lack of efficacy as well as the effects of insecticides on humans and the environment. Botanical repellents are an option that has not been fully investigated for the management of these pests. Here, we investigated the behavioral responses of common urban pests of the southwestern USA to recently discovered coconut fatty acids (CFAs), to establish the potential use of these compounds as repellents. RESULTS: Fresh residues of CFA mixture (CFAm) and their constituents caprylic acid, capric acid, capric acid methyl ester, lauric acid, and lauric acid methyl ester, tested at a concentration of 1 mg cm-2 , strongly repelled all arthropods. The repellent activity of CFAm lasted for at least 7 days, and the addition of lavender oil, used as an odor-masking agent, did not decrease this effect. Concentrations of CFAm ten times lower (0.1 mg cm-2 ) still repelled Turkestan cockroaches, and concentrations 100 times lower (0.01 mg cm-2 ) repelled T. rubida and scorpions. CONCLUSIONS: CFAm and some of their constituents are efficacious, economical, and logistically feasible for inclusion in integrated pest management programs for these important urban pests of the southwestern USA. © 2023 Society of Chemical Industry.

Effects of sex ratio on different biological parameters of Engytatus varians (Distant) (Hemiptera: Miridae) adults and their offspring: prey preference for Bactericera cockerelli (Sulcer) (Hemiptera: Triozidae).

In the present study, the influence of three sex ratios (1:1, 1:2, and 1:3; female:male) of the mirid Engytatus varians (Distant) (Hemiptera) on different biological parameters and on its offspring was evaluated. The prey preference of different developmental stages of this predator for different nymphal instars (N) of Bactericera cockerelli (Sulcer) (Hemiptera: Triozidae) was also evaluated. The fertility was significantly higher (24 nymphs/female) in the 1:3 sex ratio than in the 1:1 and 1:2 sex ratios (14 and 16 nymphs/female, respectively). The females in the 1:1 and 1:2 sex ratios lived 1.14 and 1.43 days more (27 and 28 days, respectively) than those in the 1:3 sex ratio (26 days). The nymphs derived from the females of the three sex ratios (first filial generation, F1) had five instars and a duration of 17 or 18 days. The ratio of the F1 generation females was not affected by the sex ratio of their parents. In choice tests, independent of whether the preys were placed on a single or multiple tomato (Solanum lycopersicum L.) leaflets, the consumption of females and males and N3, N4, and N5 nymphs of E. varians on B. cockerelli, generally showed the order of N2>N3>N4>N5. In conclusion, the findings revealed in this study can help to improve the rearing methodology for increasing populations of E. varians. In addition, they can serve as a guideline for releasing this predator in times when there is an abundance of early instar nymphs of B. cockerelli.

Multiple-Choice Tests: A-Z in Best Writing Practices.

Multiple-choice tests are the most used method of assessment in medical education. However, there is limited literature in medical education and psychiatry to inform the best practices in writing good-quality multiple-choice questions. Moreover, few physicians and psychiatrists have received training and have experience in writing them. This article highlights the strategies in writing high-quality multiple-choice items and discusses some common flaws that can impact validity and reliability of the assessment examinations.

Males can't afford to be choosy: Male reproductive investment does not influence preference for female size in Limia (Poeciliidae).

Reproductive investment was initially thought to be the key selective force behind male mate preference for female characters, like size or ornamentation (i.e., the preference by males for certain females). Yet, evidence of polygynous species, where male reproductive investments are often inexpensive compared to those of females, have also been described to possess male preference. Our study aims to understand how reproductive investment influences the selection of choosy males in polygynous systems using two species of livebearing fishes varying in reproductive investment: Limia perugiae, in which males invest heavily into reproduction, and L. zonata in which males invest minimally into reproduction. We hypothesized that male reproductive investment when combined with fecundity selection will favour the evolution of male mate preference and thereby lead to males that invest heavily into reproduction being choosier than males that invest minimally. When male Limia were exposed to two females simultaneously, one from the small size class and one from the large size class, L. zonata chose large females over small ones, whereas L. perugiae allocated the same amount of time regardless of female size class. Although we fail to find support for our original hypotheses, our study highlights the need for a more thorough examination of non-model species like Limia. We suggest future studies analyse reproductive investment as it interacts with cryptic choice, multiple sensory cues as well as expand comparisons to multiple Limia species, especially those endemic to Cuba, Grand Cayman, and Jamaica.

Notes From the Field: Automatic Item Generation, Standard Setting, and Learner Performance in Mastery Multiple-Choice Tests.

Mastery learning assessments have been described in simulation-based educational interventions; however, studies applying mastery learning to multiple-choice tests (MCTs) are lacking. This study investigates an approach to item generation and standard setting for mastery learning MCTs and evaluates the consistency of learner performance across sequential tests. Item models, variables for question stems, and mastery standards were established using a consensus process. Two test forms were created using item models. Tests were administered at two training programs. The primary outcome, the test-retest consistency of pass-fail decisions across versions of the test, was 94% (κ = .54). Decision-consistency classification was .85. Item-level consistency was 90% (κ = .77, SE = .03). These findings support the use of automatic item generation to create mastery MCTs which produce consistent pass-fail decisions. This technique broadens the range of assessment methods available to educators that require serial MCT testing, including mastery learning curricula.

Oviposition Behavior and Development of Aster Leafhoppers (Hemiptera: Cicadellidae) on Selected Host Plants From the Canadian Prairies.

Some plant pathogens are capable of manipulating their insect vectors and plant hosts in a way that disease transmission is enhanced. Aster leafhopper (Macrosteles quadrilineatus Forbes) (Hemiptera: Cicadellidae) is the main vector of Aster Yellows Phytoplasma (Candidatus Phytoplasma asteris) in the Canadian Prairies, which causes Aster Yellows (AY) disease in over 300 plant species including cereals and oilseeds. However, little is known about the host range of Aster leafhoppers or their host-choice selection behavior in this geographical region. Several crop and noncrop species commonly found in the Canadian Prairies were evaluated as food and reproductive hosts for Aster leafhoppers through no-choice bioassays. To study possible effects of pathogen infection, AY-uninfected and AY-infected insects were used. Cereals and some noncrops like fleabane were suitable reproductive hosts for Aster leafhoppers, with numbers of offspring observed in treatments using both AY-uninfected and AY-infected insects, suggesting an egg-laying preference on these plant species. Development was similar across the different plant species, except for canola and sowthistle, where growth indexes were lower. Sex-ratios of Aster leafhopper adults did not differ among the plant species or with respect to AY infection. Potential fecundity differed across plant species and was affected by the infection status of the insect. These findings have implications for AY epidemiology and suggest that while cereals can be suitable host plants for Aster leafhopper oviposition and development, some noncrop species could act as alternate hosts for leafhoppers that migrate into the Canadian Prairies before emergence of cereal and canola crops.

Phytoplasma Infection of Cranberry Affects Development and Oviposition, but Not Host-Plant Selection, of the Insect Vector Limotettix vaccinii.

Vector-borne pathogens, such as phytoplasmas, are known to manipulate both host plants and insect vectors to enhance their own transmission. In cranberries, phytoplasma infection causes false blossom disease, which is vectored by blunt-nosed leafhoppers (Limotettix vaccinii Van Duzee). We explored how phytoplasma infection of vegetative cranberry tissues affects the developmental performance and host preferences of L. vaccinii, and compared volatile emissions and phytohormone levels of infected and uninfected plants. In no-choice performance assays, L. vaccinii survival was similar on infected and uninfected cranberry plants. However, nymphs on infected plants took longer to reach adulthood, and produced larger adults, than those on uninfected plants. In oviposition tests, L. vaccinii females laid more eggs on uninfected than phytoplasma-infected cranberry plants. Olfactometer studies revealed preferences of L. vaccinii nymphs and adults for odors of uninfected plants when compared to clean air but not when compared to infected plants. Uninfected plants emitted higher amounts of volatiles than infected plants, particularly (E)-4,8-dimethyl-1, 3, 7-nonatriene, dodecane, and germacrene-D; however, we observed no effect of infection on phytohormone levels. Overall, our data show an oviposition preference of L. vaccinii for healthy plants, whereas our attraction data show no preference between infected and uninfected plants. While slower development on phytoplasma-infected cranberry plants led to larger adult size, there was no evidence for any beneficial effects of infection on other L. vaccinii performance parameters. Results from these studies are of relevance to understand tripartite plant-phytoplasma-insect vector interactions and are discussed in the context of the vector manipulation hypothesis.

Choice test to determine potential attractants and repellents for the sheep scab mite, Psoroptes ovis (Acari: Psoroptidae).

A choice test bioassay was devised to screen compounds as potential semiochemicals (e.g., kairomones or allomones that mediate aggregation, attraction or repellence) for the obligate parasitic mite, Psoroptes ovis. The choice test used filter paper discs in a 1:4 test:control ratio and was found to be a reliable, effective and efficient method. Four mammalian lipid components were assessed as potential attractants-linoleic acid, arachidonic acid, methyl myristate and squalene-, and the insect/tick repellent DEET for potential repellence. Linoleic acid was significantly attractive to P. ovis adult females and has the potential to act as an attractant. Identification of P. ovis semiochemicals, especially attractants, would be beneficial in the development of novel control methods and tools for this species. This is essential considering the increase in resistance to the limited prophylactic chemical treatments in the UK, and the high prevalence of scab infections.

[Formula: see text]Determination of the smoking gun of intent: significance testing of forced choice results in social security claimants.

OBJECTIVE: Significantly below-chance findings on forced choice tests have been described as revealing "the smoking gun of intent" that proved malingering. The issues of probability levels, one-tailed vs. two-tailed tests, and the combining of PVT scores on significantly below-chance findings were addressed in a previous study, with a recommendation of a probability level of .20 to test the significance of below-chance results. The purpose of the present study was to determine the rate of below-chance findings in a Social Security Disability claimant sample using the previous recommendations. METHOD: We compared the frequency of below-chance results on forced choice performance validity tests (PVTs) at two levels of significance, .05 and .20, and when using significance testing on individual subtests of the PVTs compared with total scores in claimants for Social Security Disability in order to determine the rate of the expected increase. RESULTS: The frequency of significant results increased with the higher level of significance for each subtest of the PVT and when combining individual test sections to increase the number of test items, with up to 20% of claimants showing significantly below-chance results at the higher p-value. CONCLUSIONS: These findings are discussed in light of Social Security Administration policy, showing an impact on policy issues concerning child abuse and neglect, and the importance of using these techniques in evaluations for Social Security Disability.

Analyzing Test-Taking Behavior: Decision Theory Meets Psychometric Theory.

We investigate the implications of penalizing incorrect answers to multiple-choice tests, from the perspective of both test-takers and test-makers. To do so, we use a model that combines a well-known item response theory model with prospect theory (Kahneman and Tversky, Prospect theory: An analysis of decision under risk, Econometrica 47:263-91, 1979). Our results reveal that when test-takers are fully informed of the scoring rule, the use of any penalty has detrimental effects for both test-takers (they are always penalized in excess, particularly those who are risk averse and loss averse) and test-makers (the bias of the estimated scores, as well as the variance and skewness of their distribution, increase as a function of the severity of the penalty).

Análisis factorial de ítems de respuesta forzada: una revisión y un ejemplo / Factor analysis of forced-choice items: A review and an example

Los test de respuesta forzada son ampliamente utilizados para reducir el efecto de los diferentes tipos de sesgo en la respuesta asociados a los test psicológicos (por ejemplo, la aquiescencia o la deseabilidad social). Sin embargo, este tipo de test genera los denominados datos ipsativos, los cuales poseen propiedades que hacen desaconsejable la aplicación de las técnicas clásicas de análisis factorial para su evaluación psicométrica. Pese a ello, en la práctica, muchos investigadores siguen empleando estos procedimientos para analizar los ítems de repuesta forzada; esto lleva necesariamente a conclusiones erróneas. El presente trabajo expone las propiedades analíticas de los ítems de respuesta forzada, así como un ejemplo que ilustra cómo afectan estas propiedades a la aplicación de las técnicas estadísticas clásicas y conducen a interpretaciones erróneas. Adicionalmente, se presenta una de las principales alternativas para analizar este tipo de datos basada en el modelo de juicio comparativo de Thurstone, así como los resultados de un estudio de simulación que ilustra su aplicación y efectividad en la recuperación de la estructura factorial original.

Forced-choice tests are widely used in order to reduce the impact of different response set biases typically associated to psychological tests (e.g. acquiescence or social desirability). However, these tests produce ipsative data which have undesirable properties, thereby making an inappropriate application of classical factor analysis techniques for psychometric evaluation commonly used by researchers. This paper explains the analytical properties of forced-choice tests, along with an example that illustrates how these properties have an impact on the application of conventional statistical techniques and produce improper results. Additionally, one of the current proposals is presented in order to analyze these data based on the comparative judgment model by Thurstone, along with the results of a simulation study which illustrates its implementation and effectiveness in recovering the original factor structure.

Host preference and suitability of grasses for Oebalus pugnax.

The rice stink bug, Oebalus pugnax (Fabricius) (Hemiptera: Pentatomidae: Carpocorini), though graminaceous, discriminates among its numerous host grass species. This could represent a feeding preference, it could be related to host suitability for growth and development. To clarify the role of host grass discrimination, two laboratory studies were conducted: (1) free-choice tests to evaluate preferences of O. pugnax among 11 wild host grass species found in three rice-producing counties of the central Mississippi Delta (MS, USA), and (2) no-choice tests to evaluate the impact of rice (Oryza sativa L.), junglerice [Echinochloa colona (L.) Link], and dallisgrass (Paspalum dilatatum Poir.) (all Poaceae), on the development of O. pugnax from second instar to adult. In the free-choice test, four experiments were conducted, each with four sets of host grass species and observed 1, 2, 4, 8, and 16 h after release in cages. Approximately 4 h was necessary for O. pugnax to settle on preferred host grasses. Oebalus pugnax showed a feeding preference for junglerice over all 10 other grass species. Bahiagrass, Paspalum notatum Flueggé, was the least preferred. The no-choice tests showed significant effect of host grass species on O. pugnax mean development time of nymphal survival to adults. Survival of nymphs was lower and mean development time was longer on dallisgrass compared to rice and junglerice. Knowledge of O. pugnax rate of growth and development on host grasses could be useful in the future development of rice integrated pest management strategies.

DISENTANGLING THE EFFECTS OF MATING PROPENSITY AND MATING CHOICE IN DROSOPHILA.

Incipient sexual isolation between genotypes, lines, or populations of the same species is commonly measured in Drosophila by choice tests. Results of these tests are known to be influenced, in an undetermined manner, by the mating propensity of competitors and by discriminatory factors during courtship. We have approached the problem by measuring male and female propensities in separate, independent tests, and by examining whether these estimates could explain the results of the choice tests. First, male and female choice tests were used to measure sexual isolation between populations of Drosophila melanogaster and between populations of D. simulans. Significant deviations from random mating occurred in 31 out of 48 tests, in agreement with the propensity values of the tested genotypes. We conclude that mating propensity instead of discrimination is directly involved in the estimation of sexual isolation in our populations, and advise against the application of male and female choice tests to assess intraspecific isolation without a proper knowledge of the mating propensities of competing individuals. Second, multiple choice tests were used to assess isolation between D. melanogaster populations. In examining the dynamics of matings throughout the test, we show that if competing individuals differ in mating propensities and tests are long enough to allow most matings to happen, a spurious sexual isolation can appear. We recommend that multiple choice tests be terminated once 50 percent of matings had been observed.

The influence of proline on diet selection: sex-specific feeding preferences by the grasshoppers Ageneotettix deorum and Phoetaliotes nebrascensis (Orthoptera: Acrididae).

Diet selection based on the level of proline in an insect's host plant has been observed for a number of phytophagous insects, but few studies have examined potential differences in feeding preferences between males and females. The level of proline among an insect's host plants, particularly in drought-stressed plants, can be highly variable and often is positively correlated with soluble nitrogen levels. Additionally, proline is known to participate in a number of physiological functions in insects. We tested the effect of proline as a feeding stimulant in reproductively active grasshoppers using the graminivorous Ageneotettix deorum and the generalist, but mostly graminivorous, Phoetaliotes nebrascensis. Feeding preference tests using diets with representative free amino acid and sucrose levels but varying proline levels (zero, normal and 3 x normal) were examined. The feeding preference exhibited by both species was sex-specific, although the sex-specific response was more pronounced in P. nebrascensis than in A. deorum. Females of both species displayed preferences for diets high in proline. Males of neither species exhibited a preference for proline when responses were averaged over all treatment levels. However, within specific treatment combinations, male A. deorum preferred diets with high proline over diets with zero proline. These results suggest that diet selection for specific nutrients may vary between males and females because of differences in their physiological status and, possibly, differences in the nutritional requirements associated with reproduction. These results also suggest that subtle shifts in the concentration of individual nutrients within an insect's host plant may greatly influence insect feeding patterns.