Exposing omitted moderators: Explaining why effect sizes differ in the social sciences.

Policymakers increasingly rely on behavioral science in response to global challenges, such as climate change or global health crises. But applications of behavioral science face an important problem: Interventions often exert substantially different effects across contexts and individuals. We examine this heterogeneity for different paradigms that underlie many behavioral interventions. We study the paradigms in a series of five preregistered studies across one in-person and 10 online panels, with over 11,000 respondents in total. We find substantial heterogeneity across settings and paradigms, apply techniques for modeling the heterogeneity, and introduce a framework that measures typically omitted moderators. The framework's factors (Fluid Intelligence, Attentiveness, Crystallized Intelligence, and Experience) affect the effectiveness of many text-based interventions, producing different observed effect sizes and explaining variations across samples. Moderators are associated with effect sizes through two paths, with the intensity of the manipulation and with the effect of the manipulation directly. Our results motivate observing these moderators and provide a theoretical and empirical framework for understanding and predicting varying effect sizes in the social sciences.

An Electronic Teen Questionnaire, the eTeenQ, for Risk Behavior Screening During Adolescent Well Visits in an Integrated Health System: Development and Pilot Implementation.

Background: Screening for risk behaviors is a routine and essential component of adolescent preventive health visits. Early identification of risks can inform targeted counseling and care. If stored in discrete fields in the electronic health record (EHR), adolescent screening data can also be used to understand risk behaviors across a clinic or health system or to support quality improvement projects. Objective: Goals of this pilot study were to adapt and implement an existing paper adolescent risk behavior screening tool for use as an electronic data capture tool (the eTeenQ), to evaluate acceptance of the eTeenQ, and to describe the prevalence of the selected risk behaviors reported through the eTeenQ. Methods: The multidisciplinary project team applied an iterative process to develop the 29-item eTeenQ. Two unique data entry forms were created with attention to (1) user interface and user experience, (2) the need to maintain patient privacy, and (3) the potential to transmit and store data for future use in clinical care and research. Three primary care clinics within a large health system piloted the eTeenQ from August 17, 2020, to August 27, 2021. During preventive health visits for adolescents aged 12 to 18 years, the eTeenQ was completed on tablets and responses were converted to a provider display for teens and providers to review together. Responses to the eTeenQ were stored in a REDCap (Research Electronic Data Capture; Vanderbilt University) database, and for patients who agreed, responses were transferred to an EHR flowsheet. Responses to selected eTeenQ questions are reported for those consenting to research. At the conclusion of the pilot, the study team conducted semistructured interviews with providers and staff regarding their experience using the eTeenQ. Results: Among 2816 adolescents with well visits, 2098 (74.5%) completed the eTeenQ. Of these, 1811 (86.3%) agreed to store responses in the EHR. Of 1632 adolescents (77.8% of those completing the eTeenQ) who consented for research and remained eligible, 1472 (90.2%) reported having an adult they can really talk to and 1510 (92.5%) reported feeling safe in their community, yet 401 (24.6%) reported someone they lived with had a gun and 172 (10.5%) reported having had a stressful or scary event that still bothered them. In addition, 157 (9.6%) adolescents reported they were or wondered if they were gay, lesbian, bisexual, pansexual, asexual, or other, and 43 (2.6%) reported they were or wondered if they were transgender or gender diverse. Of 11 staff and 7 providers completing interviews, all felt that the eTeenQ improved confidentiality and willingness among adolescents to answer sensitive questions. All 7 providers preferred the eTeenQ over the paper screening tool. Conclusions: Electronic capture of adolescent risk behaviors is feasible in a busy clinic setting and well accepted among staff and clinicians. Most adolescents agreed for their responses to risk behavior screening to be stored in the EHR.

Identifying amyotrophic lateral sclerosis through interactions with an internet search engine.

INTRODUCTION/AIMS: Amyotrophic lateral sclerosis (ALS), a motor neuron disease, remains a clinical diagnosis with an average time from onset of symptoms to diagnosis of about 1 year. Herein we examine the possibility that interactions with an internet search engine can identify people with ALS. METHODS: We identified 285 anonymous Bing users whose queries indicated that they had been diagnosed with ALS and matched them to: (1) 3276 control users; and (2) 1814 users whose searches indicated they had ALS disease mimics. We tested whether the ALS group could be distinguished from controls and disease mimics based on search engine query data. Finally, we conducted a prospective validation from participants who provided access to their Bing search data. RESULTS: The model distinguished between the ALS group and controls with an area under the curve (AUC) of 0.81. Model scores for the ALS group differed from the disease mimics group (rank sum test, p < .05 with Bonferroni correction). Mild cognitive impairment could not be distinguished from ALS (p > .05). In the prospective analysis, the model reached an AUC of 0.74. DISCUSSION: Our results suggest that interactions with search engines should be further studied to understand the potential to act as a tool to assist in screening for ALS and to reduce diagnostic delay.

A new online paradigm to measure spontaneous pointing in infants and caregivers.

Index-finger pointing is a milestone in the development of referential communication. Previous research has investigated infants' pointing with a variety of paradigms ranging from parent reports to field observations to experimental settings, suggesting that lab-based semi-natural interactional settings seem especially suited to elicit and measure infant pointing. With the Covid-pandemic the need for a comparable online tool became evident enabling also efficient, low-cost, large-scale, diverse data collection. The current study introduces a remote online paradigm, based on the established live 'decorated-room' paradigm. In Experiment 1, 12-months old infants and their caregivers (N = 24) looked at digitally presented stimuli together while being recorded with their webcam. We coded pointing gestures of infants and caregivers as well as caregivers' responses to infants' pointing. In Experiment 2 (N = 47), we optimized stimuli and investigated influences of stimulus characteristics. We systematically varied the style of depiction, stimulus complexity, motion, and facial stimuli. Main findings were that infants and caregivers pointed spontaneously, with mean behaviors ranging within the benchmarks of previously reported findings of the live decorated-room paradigm. Further, the social setting was preserved as revealed by significant relations between parents' responsive points and infants' pointing frequency. Analyses of stimuli characteristics revealed that infants pointed more to stimuli depicting faces than to other stimuli. The new remote online paradigm proves a useful addition to established paradigms. It offers novel opportunities for simplified assessments, large-scale sampling, and worldwide, diversified data collection.

It takes two (seconds): decreasing encoding time for two-choice functional near-infrared spectroscopy brain-computer interface communication.

Significance: Brain-computer interfaces (BCIs) can provide severely motor-impaired patients with a motor-independent communication channel. Functional near-infrared spectroscopy (fNIRS) constitutes a promising BCI-input modality given its high mobility, safety, user comfort, cost-efficiency, and relatively low motion sensitivity. Aim: The present study aimed at developing an efficient and convenient two-choice fNIRS communication BCI by implementing a relatively short encoding time (2 s), considerably increasing communication speed, and decreasing the cognitive load of BCI users. Approach: To encode binary answers to 10 biographical questions, 10 healthy adults repeatedly performed a combined motor-speech imagery task within 2 different time windows guided by auditory instructions. Each answer-encoding run consisted of 10 trials. Answers were decoded during the ongoing experiment from the time course of the individually identified most-informative fNIRS channel-by-chromophore combination. Results: The answers of participants were decoded online with an accuracy of 85.8% (run-based group mean). Post-hoc analysis yielded an average single-trial accuracy of 68.1%. Analysis of the effect of number of trial repetitions showed that the best information-transfer rate could be obtained by combining four encoding trials. Conclusions: The study demonstrates that an encoding time as short as 2 s can enable immediate, efficient, and convenient fNIRS-BCI communication.

iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.

Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing "LEFT" versus "RIGHT" and "ON" versus "OFF" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.

Online dynamic flat-field correction for MHz microscopy data at European XFEL.

The high pulse intensity and repetition rate of the European X-ray Free-Electron Laser (EuXFEL) provide superior temporal resolution compared with other X-ray sources. In combination with MHz X-ray microscopy techniques, it offers a unique opportunity to achieve superior contrast and spatial resolution in applications demanding high temporal resolution. In both live visualization and offline data analysis for microscopy experiments, baseline normalization is essential for further processing steps such as phase retrieval and modal decomposition. In addition, access to normalized projections during data acquisition can play an important role in decision-making and improve the quality of the data. However, the stochastic nature of X-ray free-electron laser sources hinders the use of standard flat-field normalization methods during MHz X-ray microscopy experiments. Here, an online (i.e. near real-time) dynamic flat-field correction method based on principal component analysis of dynamically evolving flat-field images is presented. The method is used for the normalization of individual X-ray projections and has been implemented as a near real-time analysis tool at the Single Particles, Clusters, and Biomolecules and Serial Femtosecond Crystallography (SPB/SFX) instrument of EuXFEL.

Raynals, an online tool for the analysis of dynamic light scattering.

Dynamic light scattering (DLS) is routinely employed to assess the homogeneity and size-distribution profile of samples containing microscopic particles in suspension or solubilized polymers. In this work, Raynals, user-friendly software for the analysis of single-angle DLS data that uses the Tikhonov-Phillips regularization, is introduced. Its performance is evaluated on simulated and experimental data generated by different DLS instruments for several proteins and gold nanoparticles. DLS data can easily be misinterpreted and the simulation tools available in Raynals allow the limitations of the measurement and its resolution to be understood. It was designed as a tool to address the quality control of biological samples during sample preparation and optimization and it helps in the detection of aggregates, showing the influence of large particles. Lastly, Raynals provides flexibility in the way that the data are presented, allows the export of publication-quality figures, is free for academic use and can be accessed online on the eSPC data-analysis platform at https://spc.embl-hamburg.de/.

Recruitment and Retention Strategies for Parents Experiencing Vulnerability: Lessons from an Online Parenting Program.

BACKGROUND: Supportive parenting programs can promote parent-child interactions and children's development. However, families experiencing vulnerability (e.g., low socioeconomic status) report barriers (e.g., transportation, distrust of researchers) to research participation, and attrition rates of 40% and higher have been reported in parenting research. In response, we conducted a longitudinal evaluation of a digital parenting program in a major metropolitan centre in western Canada and retained 99% of our sample. AIM: Review recruitment and retention strategies employed in the First Pathways study and evaluate associations between sociodemographic (e.g., income) and psychosocial (e.g., parental depression) factors with recruitment and retention strategies. METHODS AND FINDINGS: In collaboration with community agencies, we commenced recruitment of 100 families experiencing vulnerability (e.g., low-income) in June 2021. We utilized strategies to engage staff (e.g., presentations, gift cards, updates) and employed snowball sampling. Families recruited through community agencies were significantly more likely to experience vulnerability (e.g., low income and education, high adverse experiences) compared to families in the snowball sample. We incorporated strategies to minimize participant burden (e.g., choice of online or in-person meetings), promoted rapport (e.g., holiday texts, nonjudgmental environment), incorporated trauma-informed practices (e.g., sensitive inquiry), and demonstrated appreciation for participants' contributions (i.e., honorarium). Family experiences of vulnerability (i.e., low income, depressive symptoms, adversity) were correlated with higher participant rescheduling. CONCLUSION: Nurses need knowledge of strategies to promote equitable access to research for families experiencing vulnerability. Digital programs with protocols designed to establish rapport, include trauma-informed practices, and minimize participant burden will likely optimize participation and retention.

Dematerialized participation challenges: Methods and practices for online focus groups.

This study explores the limitations and benefits of different approaches to conducting online focus groups and illustrates an online focus group protocol used within the Value for Schools project in Italy. According to the project evaluation design, 13 online focus groups were organized, with the participation of 101 teachers and 37 school principals. The protocol setup, incorporation, and reorganization of the indications have been discussed in the literature, addressing the methodological and practical issues, such as the selection of participants and preliminary communication with them; the web conference platform (Zoom Business); timing, as well as access times and mode; the roles of the researchers involved (moderator, co-host technical assistant, co-host-observer, co-host-animator) and their integration spaces; technological support; and animation tools. The recording and transcription tools and subsequent analysis of the textual corpus are presented. Finally, the authors discuss the validation and reliability of online focus group protocols.

Towards Real-Time Analysis of Gas-Liquid Pipe Flow: A Wire-Mesh Sensor for Industrial Applications.

Real-time monitoring of gas-liquid pipe flow is highly demanded in industrial processes in the chemical and power engineering sectors. Therefore, the present contribution describes the novel design of a robust wire-mesh sensor with an integrated data processing unit. The developed device features a sensor body for industrial conditions of up to 400 °C and 135 bar as well as real-time processing of measured data, including phase fraction calculation, temperature compensation and flow pattern identification. Furthermore, user interfaces are included via a display and 4…20 mA connectivity for the integration into industrial process control systems. In the second part of the contribution, we describe the experimental verification of the main functionalities of the developed system. Firstly, the calculation of cross-sectionally averaged phase fractions along with temperature compensation was tested. Considering temperature drifts of up to 55 K, an average deviation of 3.9% across the full range of the phase fraction was found by comparison against image references from camera recordings. Secondly, the automatic flow pattern identification was tested in an air-water two-phase flow loop. The results reveal reasonable agreement with well-established flow pattern maps for both horizontal and vertical pipe orientations. The present results indicate that all prerequisites for an application in industrial environments in the near future are fulfilled.

Mining and analysing online social networks: Studying the dynamics of digital peer support.

In recent years, the rapid growth of user-generated content has led to much research evaluating the patterns of online information exchange. These studies demonstrate that online communities are valuable data sources which provide rich, longitudinal data that would otherwise be difficult, if not impossible to access. Given the increased research interest, mining and analysing online social networks has become an important research domain, encompassing a variety of approaches. To analyse the large number of observations commonly found in online communities, we propose to first mine the data using a so-called Webscraper and then combine Social Network Analysis (SNA) with Sentiment Analysis to explore both content and relationships. The hands-on approach described in this article is targeted at researchers without a background in technical disciplines. Instead of focusing on some of the specific algorithms that facilitate the mining and analysis of online data, we describe how to use and combine out-of-the-box solutions to collect and analyse the online network data. Moreover, we document the steps taken and present important lessons learnt throughout the process of collecting and analysing data from an online health community with 108,569 registered users who contributed to 197,980 discussions with a total of 484,250 replies. In sum, our method proposes to:•Extract all relevant data from an openly accessible online community using a Webscraper.•Determine and visualise the relationships between users and the properties of the social network as a whole using Social Network Analysis.•Conduct Sentiment Analysis to detect the emotional tone of the online contributions, and to possibly infer further variables from the text such as the personality characteristics of users.

Measuring event segmentation: An investigation into the stability of event boundary agreement across groups.

People spontaneously divide everyday experience into smaller units (event segmentation). To measure event segmentation, studies typically ask participants to explicitly mark the boundaries between events as they watch a movie (segmentation task). Their data may then be used to infer how others are likely to segment the same movie. However, significant variability in performance across individuals could undermine the ability to generalize across groups, especially as more research moves online. To address this concern, we used several widely employed and novel measures to quantify segmentation agreement across different sized groups (n = 2-32) using data collected on different platforms and movie types (in-lab & commercial film vs. online & everyday activities). All measures captured nonrandom and video-specific boundaries, but with notable between-sample variability. Samples of 6-18 participants were required to reliably detect video-driven segmentation behavior within a single sample. As sample size increased, agreement values improved and eventually stabilized at comparable sample sizes for in-lab & commercial film data and online & everyday activities data. Stabilization occurred at smaller sample sizes when measures reflected (1) agreement between two groups versus agreement between an individual and group, and (2) boundary identification between small (fine-grained) rather than large (coarse-grained) events. These analyses inform the tailoring of sample sizes based on the comparison of interest, materials, and data collection platform. In addition to demonstrating the reliability of online and in-lab segmentation performance at moderate sample sizes, this study supports the use of segmentation data to infer when events are likely to be segmented.

User perspectives on long-term remote active electronic self-monitoring of mood symptoms in bipolar spectrum disorders.

BACKGROUND: User feedback is crucial in the development of electronic self-monitoring tools for bipolar spectrum disorders (BSD). Previous studies have examined user experiences in small samples self-monitoring over relatively short time periods. We aimed to explore the experiences of a large sample of individuals with BSD engaged in long-term remote active electronic self-monitoring. METHODS: An online survey, containing closed and open questions, was sent to participants with BSD enrolled on the Bipolar Disorder Research Network (BDRN) True Colours mood-monitoring system. Questions related to experiences of using True Colours, including viewing mood graphs, and sharing data with healthcare professionals (HCPs) and/or family/friends. RESULTS: Response rate was 62.7 % (n = 362). 88.4 % reported finding using True Colours helpful. Commonly reported benefits were having a visual record of mood changes, patterns/triggers and identifying early warning signs. Limitations included questions not being comprehensive or revealing anything new. One third had shared their graphs, with 89.9 % finding it helpful to share with HCPs and 78.7 % helpful to share with family/friends. Perceived benefits included aiding communication and limitations included lack of interest/understanding from others. LIMITATIONS: Responder bias may be present. Findings may not be generalisable to all research cohorts. CONCLUSIONS: The majority of participants valued long-term self-monitoring. Personalisation and ease of use were important. A potential challenge is continued use when mood is long-term stable, highlighting the need for measures to be sensitive to small changes. Sharing self-monitoring data with HCPs may enhance communication of the lived experience of those with BSD. Future research should examine HCPs' perspectives.

HRQoL and nutritional well-being dissimilarities between two different online collection methods: Value for digital health implementation.

Background: Online health data collection has gained a reputation over the last years to record and process information about health issues for implementing digital health. Objective: The research aim was to appraise two online methods (open and rewarded) to collect information about HRQoL and nutritional well-being and to compare the results between both surveyed populations. Methods: This cross-sectional study is framed on the NUTRiMDEA project. Online data through two different web-based methods (open survey and rewarded survey) were retrieved to assemble data related to sociodemographic, lifestyle (diet, physical activity and sleep patterns) and general health aspects, as well as HRQoL by an evidence-based form such as the SF-12 questionnaire, the IPAQ survey, and MEDAS-14, participants were adults (>18 years old). Results: Overall, 17,332 participants responded to the open survey (OS, n = 11,883) or the rewarded survey (RS, n = 5449). About 65.1% of the participants were female, while the mean age was in the range of 40-70 years. There were significant differences (p < 0.05) between surveyed populations in sociodemographic, lifestyle (diet and physical activity), health and HRQoL data. Conclusions: This investigation implemented an evidence-based online questionnaire that collected demographic, lifestyle factors, phenotypic and health-related aspects as well as compared differential outcomes in HRQoL and nutritional/lifestyle well-being depending on the online mode data collection. Findings demonstrated dissimilarities in most aspects of health, HRQoL, dietary intake and physical activity records between both populations. Overall, OS sample was characterized as a healthier population with superior lifestyle habits than RS participants.

New data analysis for BioSAXS at the ESRF.

The second phase of the ESRF upgrade program did not only provide a new storage ring (Extremely Brilliant Source, EBS) but also allowed several beamlines to be refurbished. The BioSAXS beamline (located on port BM29) was upgraded with a new wiggler source and a larger detector. All analysis software has been rewritten to cope with the increased data flux and continues to provide beamline users with reduced and pre-processed data in real time. This article describes FreeSAS, an open-source collection of various small-angle scattering analysis algorithms needed to reduce and analyze BioSAXS data, and Dahu, the tool used to interface data analysis with beamline control. It further presents the data-processing pipelines for the different data acquisitions modes of the beamline, using either a sample changer for individual homogeneous samples or an inline size-exclusion chromatography setup.

Researching Alcohol Consumption During Pregnancy. Opportunities and Challenges With Two Methods of Data Acquisition.

Qualitative research methods, primarily interviews, have become more common in the field of addiction research. Other data sources were often neglected, although sources such as social media can offer insights into the realities of people, since social media also plays a relevant role in today's living environments. This article examines the use of online forums as an underutilized data source in contrast to telephone interviews, to identify methodological opportunities and challenges. We analyzed nine discussion threads and seven interviews about 'alcohol consumption during pregnancy'. Discursive comparison of the results was performed with a focus on sampling issues, comparability and risks for participants and researchers. Key issues were present in both data sources. People with different opinions were openly hostile in forums, while tolerance was more often expressed in interviews. The interviews showed a rather mild communication style, which could be attributed to social desirability. In the forum discussions, the participants often expressed themselves very directly. To comprehensively grasp the subject matter of the research, it is important to recognize the types of communication promoted by different data sources. These results have implications for research about female substance use. Knowledge of the issues will bestow a valuable contribution to researchers working in the field of substance use to help develop appropriate research approaches, as they engage in research into this highly stigmatized and controversial area.

Doing Experimental Psychological Research from Remote: How Alerting Differently Impacts Online vs. Lab Setting.

Due to pandemic-imposed restrictions on lab-based research, we have recently witnessed a flourishing of online studies in experimental psychology, based on the collection of fine behavioral measures such as reaction times (RTs) and accuracy. However, it remains unclear whether participants' alerting levels may have a different impact on behavioral performance in the online vs. lab setting. In this work we administered online and in-lab the dynamic temporal prediction (DTP) task, which requires an implicit modulation of participants' alerting by alternating experimental conditions implying either slower or faster response rates. We then compared data distribution, RTs, accuracy, and time-on-task effects across the adult lifespan between the settings. We replicated online and across the whole age range considered (19-69 y) all the task-specific effects already found in-lab (both in terms of RTs and accuracy) beyond the overall RTs delay typical of the online setting. Moreover, we found an interaction between the setting and task-specific features so that participants showed slower RTs only in experimental conditions implying a less urgent response rate, while no RTs delay and a slight accuracy increase emerged in faster conditions. Thus, the online setting has been shown to be methodologically sound in eliciting comparable effects to those found in-lab. Moreover, behavioral performance seems to be more sensitive to task-induced alerting shifts in the online as compared to the lab setting, leading to either a heightened or reduced efficiency depending on a faster or slower response rate of experimental conditions, respectively.