Exposing omitted moderators: Explaining why effect sizes differ in the social sciences.

Policymakers increasingly rely on behavioral science in response to global challenges, such as climate change or global health crises. But applications of behavioral science face an important problem: Interventions often exert substantially different effects across contexts and individuals. We examine this heterogeneity for different paradigms that underlie many behavioral interventions. We study the paradigms in a series of five preregistered studies across one in-person and 10 online panels, with over 11,000 respondents in total. We find substantial heterogeneity across settings and paradigms, apply techniques for modeling the heterogeneity, and introduce a framework that measures typically omitted moderators. The framework's factors (Fluid Intelligence, Attentiveness, Crystallized Intelligence, and Experience) affect the effectiveness of many text-based interventions, producing different observed effect sizes and explaining variations across samples. Moderators are associated with effect sizes through two paths, with the intensity of the manipulation and with the effect of the manipulation directly. Our results motivate observing these moderators and provide a theoretical and empirical framework for understanding and predicting varying effect sizes in the social sciences.

A new online paradigm to measure spontaneous pointing in infants and caregivers.

Index-finger pointing is a milestone in the development of referential communication. Previous research has investigated infants' pointing with a variety of paradigms ranging from parent reports to field observations to experimental settings, suggesting that lab-based semi-natural interactional settings seem especially suited to elicit and measure infant pointing. With the Covid-pandemic the need for a comparable online tool became evident enabling also efficient, low-cost, large-scale, diverse data collection. The current study introduces a remote online paradigm, based on the established live 'decorated-room' paradigm. In Experiment 1, 12-months old infants and their caregivers (N = 24) looked at digitally presented stimuli together while being recorded with their webcam. We coded pointing gestures of infants and caregivers as well as caregivers' responses to infants' pointing. In Experiment 2 (N = 47), we optimized stimuli and investigated influences of stimulus characteristics. We systematically varied the style of depiction, stimulus complexity, motion, and facial stimuli. Main findings were that infants and caregivers pointed spontaneously, with mean behaviors ranging within the benchmarks of previously reported findings of the live decorated-room paradigm. Further, the social setting was preserved as revealed by significant relations between parents' responsive points and infants' pointing frequency. Analyses of stimuli characteristics revealed that infants pointed more to stimuli depicting faces than to other stimuli. The new remote online paradigm proves a useful addition to established paradigms. It offers novel opportunities for simplified assessments, large-scale sampling, and worldwide, diversified data collection.

iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.

Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing "LEFT" versus "RIGHT" and "ON" versus "OFF" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.

Recruitment and Retention Strategies for Parents Experiencing Vulnerability: Lessons from an Online Parenting Program.

BACKGROUND: Supportive parenting programs can promote parent-child interactions and children's development. However, families experiencing vulnerability (e.g., low socioeconomic status) report barriers (e.g., transportation, distrust of researchers) to research participation, and attrition rates of 40% and higher have been reported in parenting research. In response, we conducted a longitudinal evaluation of a digital parenting program in a major metropolitan centre in western Canada and retained 99% of our sample. AIM: Review recruitment and retention strategies employed in the First Pathways study and evaluate associations between sociodemographic (e.g., income) and psychosocial (e.g., parental depression) factors with recruitment and retention strategies. METHODS AND FINDINGS: In collaboration with community agencies, we commenced recruitment of 100 families experiencing vulnerability (e.g., low-income) in June 2021. We utilized strategies to engage staff (e.g., presentations, gift cards, updates) and employed snowball sampling. Families recruited through community agencies were significantly more likely to experience vulnerability (e.g., low income and education, high adverse experiences) compared to families in the snowball sample. We incorporated strategies to minimize participant burden (e.g., choice of online or in-person meetings), promoted rapport (e.g., holiday texts, nonjudgmental environment), incorporated trauma-informed practices (e.g., sensitive inquiry), and demonstrated appreciation for participants' contributions (i.e., honorarium). Family experiences of vulnerability (i.e., low income, depressive symptoms, adversity) were correlated with higher participant rescheduling. CONCLUSION: Nurses need knowledge of strategies to promote equitable access to research for families experiencing vulnerability. Digital programs with protocols designed to establish rapport, include trauma-informed practices, and minimize participant burden will likely optimize participation and retention.

Dematerialized participation challenges: Methods and practices for online focus groups.

This study explores the limitations and benefits of different approaches to conducting online focus groups and illustrates an online focus group protocol used within the Value for Schools project in Italy. According to the project evaluation design, 13 online focus groups were organized, with the participation of 101 teachers and 37 school principals. The protocol setup, incorporation, and reorganization of the indications have been discussed in the literature, addressing the methodological and practical issues, such as the selection of participants and preliminary communication with them; the web conference platform (Zoom Business); timing, as well as access times and mode; the roles of the researchers involved (moderator, co-host technical assistant, co-host-observer, co-host-animator) and their integration spaces; technological support; and animation tools. The recording and transcription tools and subsequent analysis of the textual corpus are presented. Finally, the authors discuss the validation and reliability of online focus group protocols.

Measuring event segmentation: An investigation into the stability of event boundary agreement across groups.

People spontaneously divide everyday experience into smaller units (event segmentation). To measure event segmentation, studies typically ask participants to explicitly mark the boundaries between events as they watch a movie (segmentation task). Their data may then be used to infer how others are likely to segment the same movie. However, significant variability in performance across individuals could undermine the ability to generalize across groups, especially as more research moves online. To address this concern, we used several widely employed and novel measures to quantify segmentation agreement across different sized groups (n = 2-32) using data collected on different platforms and movie types (in-lab & commercial film vs. online & everyday activities). All measures captured nonrandom and video-specific boundaries, but with notable between-sample variability. Samples of 6-18 participants were required to reliably detect video-driven segmentation behavior within a single sample. As sample size increased, agreement values improved and eventually stabilized at comparable sample sizes for in-lab & commercial film data and online & everyday activities data. Stabilization occurred at smaller sample sizes when measures reflected (1) agreement between two groups versus agreement between an individual and group, and (2) boundary identification between small (fine-grained) rather than large (coarse-grained) events. These analyses inform the tailoring of sample sizes based on the comparison of interest, materials, and data collection platform. In addition to demonstrating the reliability of online and in-lab segmentation performance at moderate sample sizes, this study supports the use of segmentation data to infer when events are likely to be segmented.

User perspectives on long-term remote active electronic self-monitoring of mood symptoms in bipolar spectrum disorders.

BACKGROUND: User feedback is crucial in the development of electronic self-monitoring tools for bipolar spectrum disorders (BSD). Previous studies have examined user experiences in small samples self-monitoring over relatively short time periods. We aimed to explore the experiences of a large sample of individuals with BSD engaged in long-term remote active electronic self-monitoring. METHODS: An online survey, containing closed and open questions, was sent to participants with BSD enrolled on the Bipolar Disorder Research Network (BDRN) True Colours mood-monitoring system. Questions related to experiences of using True Colours, including viewing mood graphs, and sharing data with healthcare professionals (HCPs) and/or family/friends. RESULTS: Response rate was 62.7 % (n = 362). 88.4 % reported finding using True Colours helpful. Commonly reported benefits were having a visual record of mood changes, patterns/triggers and identifying early warning signs. Limitations included questions not being comprehensive or revealing anything new. One third had shared their graphs, with 89.9 % finding it helpful to share with HCPs and 78.7 % helpful to share with family/friends. Perceived benefits included aiding communication and limitations included lack of interest/understanding from others. LIMITATIONS: Responder bias may be present. Findings may not be generalisable to all research cohorts. CONCLUSIONS: The majority of participants valued long-term self-monitoring. Personalisation and ease of use were important. A potential challenge is continued use when mood is long-term stable, highlighting the need for measures to be sensitive to small changes. Sharing self-monitoring data with HCPs may enhance communication of the lived experience of those with BSD. Future research should examine HCPs' perspectives.

HRQoL and nutritional well-being dissimilarities between two different online collection methods: Value for digital health implementation.

Background: Online health data collection has gained a reputation over the last years to record and process information about health issues for implementing digital health. Objective: The research aim was to appraise two online methods (open and rewarded) to collect information about HRQoL and nutritional well-being and to compare the results between both surveyed populations. Methods: This cross-sectional study is framed on the NUTRiMDEA project. Online data through two different web-based methods (open survey and rewarded survey) were retrieved to assemble data related to sociodemographic, lifestyle (diet, physical activity and sleep patterns) and general health aspects, as well as HRQoL by an evidence-based form such as the SF-12 questionnaire, the IPAQ survey, and MEDAS-14, participants were adults (>18 years old). Results: Overall, 17,332 participants responded to the open survey (OS, n = 11,883) or the rewarded survey (RS, n = 5449). About 65.1% of the participants were female, while the mean age was in the range of 40-70 years. There were significant differences (p < 0.05) between surveyed populations in sociodemographic, lifestyle (diet and physical activity), health and HRQoL data. Conclusions: This investigation implemented an evidence-based online questionnaire that collected demographic, lifestyle factors, phenotypic and health-related aspects as well as compared differential outcomes in HRQoL and nutritional/lifestyle well-being depending on the online mode data collection. Findings demonstrated dissimilarities in most aspects of health, HRQoL, dietary intake and physical activity records between both populations. Overall, OS sample was characterized as a healthier population with superior lifestyle habits than RS participants.

Doing Experimental Psychological Research from Remote: How Alerting Differently Impacts Online vs. Lab Setting.

Due to pandemic-imposed restrictions on lab-based research, we have recently witnessed a flourishing of online studies in experimental psychology, based on the collection of fine behavioral measures such as reaction times (RTs) and accuracy. However, it remains unclear whether participants' alerting levels may have a different impact on behavioral performance in the online vs. lab setting. In this work we administered online and in-lab the dynamic temporal prediction (DTP) task, which requires an implicit modulation of participants' alerting by alternating experimental conditions implying either slower or faster response rates. We then compared data distribution, RTs, accuracy, and time-on-task effects across the adult lifespan between the settings. We replicated online and across the whole age range considered (19-69 y) all the task-specific effects already found in-lab (both in terms of RTs and accuracy) beyond the overall RTs delay typical of the online setting. Moreover, we found an interaction between the setting and task-specific features so that participants showed slower RTs only in experimental conditions implying a less urgent response rate, while no RTs delay and a slight accuracy increase emerged in faster conditions. Thus, the online setting has been shown to be methodologically sound in eliciting comparable effects to those found in-lab. Moreover, behavioral performance seems to be more sensitive to task-induced alerting shifts in the online as compared to the lab setting, leading to either a heightened or reduced efficiency depending on a faster or slower response rate of experimental conditions, respectively.

Open Lab: A web application for running and sharing online experiments.

Online experiments have become a popular way of collecting data in the social and behavioral sciences. However, the high technical hurdles of setting up a server may prevent researchers from starting them. Also, proprietary software may restrict a researcher's freedom to customize or share their study. Open Lab is a server-side application designed to host online surveys and experiments created using lab.js. Available online at https://open-lab.online , Open Lab offers a fast, secure, and transparent way to deploy studies; it handles uploading experiment scripts, customizing study design, managing the participant database, and working with the study results. Open Lab is integrated with the lab.js experiment builder ( https://lab.js.org/ ), a browser-based program which enables the creation of new studies from scratch or the use of templates. This paper compares Open Lab with other study deployment services, discusses how Open Lab contributes to open science practices, and provides a step-by-step guide for researchers.

"Do I still have epilepsy?" Epilepsy identity 15-20 years after anterior temporal lobectomy.

OBJECTIVE: Identity is a multifaceted construct, comprising personal identity (sense of being a unique individual) and social identity (the sense-of-self derived from membership of social groups). Social identity involves explicit identification with a group ("I am …") and implicit behaviors or attitudes associated with group membership. Following successful treatment with surgery, patients with epilepsy can undergo a complex and lasting change in personal identity. To date, there has been no research into postoperative social epilepsy identity (SEI). We sought to examine SEI 15-20 years post-surgery, and the relationship between SEI and satisfaction with surgery, psychosocial improvements, mood, and health-related quality of life (HRQoL). METHODS: Thirty-two patients who underwent anterior temporal lobectomy (ATL; 19 female) were recruited, with a median follow-up of 18 years (interquartile range [IQR] = 2.5). Using a novel interactive online program, we collected data on SEI, satisfaction with surgery, and perceived psychosocial improvements, alongside standardized measures of mood (Neurological Disorders Depressio Inventory-Epilepsy; Patient Health Questionnaire-Generalised Anxiety Disorder-7 item) and HRQoL (Quality of Life in Epilepsy-31 item). Non-parametric analyses were used to analyse the data. RESULTS: Twenty-five percent of patients were free of disabling seizures since surgery, yet 65% stated they no longer had epilepsy and >90% reported satisfaction with surgery. Explicitly discarding SEI was positively associated with HRQoL at long-term follow-up, over and above seizure outcome. Implicit SEI was expressed as (a) acceptance of epilepsy, (b) a sense of belonging to the epilepsy community, and (c) difficulty disclosing and discussing epilepsy. Difficulty disclosing and discussing epilepsy was associated with increased anxiety and lower HRQoL. SIGNIFICANCE: At long-term follow-up, over half of our patients reported an explicit change in SEI, which could promote better HRQoL. In contrast, difficulty with disclosure of epilepsy was associated with increased anxiety and reduced HRQoL, possibly reflecting the ongoing effects of stigma. These findings highlight the importance of understanding changes in patient social identity for promoting long-term well-being after surgery.

lab.js: A free, open, online study builder.

Web-based data collection is increasingly popular in both experimental and survey-based research because it is flexible, efficient, and location-independent. While dedicated software for laboratory-based experimentation and online surveys is commonplace, researchers looking to implement experiments in the browser have, heretofore, often had to manually construct their studies' content and logic using code. We introduce lab.js, a free, open-source experiment builder that makes it easy to build studies for both online and in-laboratory data collection. Through its visual interface, stimuli can be designed and combined into a study without programming, though studies' appearance and behavior can be fully customized using HTML, CSS, and JavaScript code if required. Presentation and response times are kept and measured with high accuracy and precision heretofore unmatched in browser-based studies. Experiments constructed with lab.js can be run directly on a local computer and published online with ease, with direct deployment to cloud hosting, export to web servers, and integration with popular data collection platforms. Studies can also be shared in an editable format, archived, re-used and adapted, enabling effortless, transparent replications, and thus facilitating open, cumulative science. The software is provided free of charge under an open-source license; further information, code, and extensive documentation are available from https://lab.js.org/ .

Lifestyle and Health-Related Quality of Life Relationships Concerning Metabolic Disease Phenotypes on the Nutrimdea Online Cohort.

Obesity, diabetes and cardiovascular events are non-communicable diseases (NCDs) directly related to lifestyle and life quality. Rises on NCDs rates are leading to increases in early deaths concerning metabolic morbidities. Health-related quality of life (HRQoL) has been described as a subjective perception about the influence of health and personal features on human well-being. This study aimed to characterize phenotypic and lifestyle roles on the occurrence of metabolic diseases and determine the potential mutual interactions and with HRQoL. Data from an online adult population (NUTRiMDEA study, n = 17,332) were used to estimate an adapted Obesogenic Score (ObS), while logistic regression analyses were fitted in order to examine relevant factors related to the prevalence of different metabolic diseases including HRQoL. Sex and age showed significant differences depending on lifestyle and metabolic health (p < 0.05). Adherence to the Mediterranean diet and physical activity showed a mutual interaction concerning ObS (p < 0.001), as well with metabolic health (p = 0.044). Furthermore, metabolic diseases showed own features related to sociodemographic and lifestyle characteristics in this population. Metabolic syndrome components may be differently influenced by diverse lifestyle or socioeconomic factors which in turn affect the perceived HRQoL. These outcomes should be taken into account individually for a precision medicine and public health purposes.

Ética em pesquisa: a utilização de levantamento on-line em psicologia / Ethics in research: the use of online survey in psychology / Ética en investigación: el uso de encuestas en línea en psicología

Pesquisas de levantamento (survey) em Psicologia utilizando coleta de dados on-line (CDO) se tornaram muito populares no Brasil devido a sua facilidade e capacidade de reunir muitos participantes. Essa nova prática, no entanto, necessita de uma metodologia própria e levanta questões éticas importantes. Problematiza-se, neste artigo, a questão referente à inexistência de diretrizes brasileiras que estabelecem as melhores práticas para seu uso, a fim de alcançar o objetivo deste estudo teórico de explorar os dilemas éticos e metodológicos referentes ao uso de CDO e propor algumas diretrizes e soluções básicas para tais questões. Serão discutidas questões sobre consentimento, sigilo e confidencialidade e qualidade dos dados. Além disso, serão apresentadas possíveis soluções e alternativas discutidas na literatura. Conclui-se que a pesquisa com uso de CDO é muito importante para o cenário brasileiro atual, mas algumas precauções devem ser tomadas a fim de garantir a proteção dos participantes e pesquisadores, bem como a qualidade dos dados.(AU)

Surveys in Psychology using Online Data Collection (ODC) have become popular in Brazil due to their ease and potential to gather many participants. This new practice, however, needs its own methodology and raises important ethical questions. This article problematizes the issue regarding the non-existence of Brazilian guidelines that establish the best practices for its use, to reach the study aim of exploring the ethical and methodological dilemmas related to the use of ODC and proposing some basic guidelines and solutions to such issues. Issues regarding consent, secrecy and confidentiality, and data quality are discussed. Also, possible solutions and alternatives discussed in the literature are presented. In conclusion, research using ODC is very important for the current Brazilian scenario, but some precautions must be taken to guarantee the protection of participants and researchers, as well as the quality of the data.(AU)

Las encuestas en línea (survey) en Psicología que utilizan Compilación de datos en línea (CDO) son muy empleadas en Brasil debido a su facilidad y potencial de reunir a muchos participantes. Esa nueva práctica necesita su propia metodología y plantea preguntas éticas importantes. Este artículo aborda la inexistencia de directrices brasileñas que establecen las mejores prácticas para su uso, con el propósito de explorar los dilemas éticos y metodológicos relacionados con el uso de CDO, y proponer algunas pautas y soluciones básicas para tales preguntas. Se discutirán temas relacionados con el consentimiento, el secreto, la confidencialidad y la calidad de los datos. Además, se presentarán posibles soluciones y alternativas discutidas en la literatura. Se concluye que la investigación con CDO es muy importante para el escenario brasileño actual, pero se deben tomar algunas medidas para garantizar la protección de los participantes e investigadores, así como la calidad de los datos.(AU)

Opportunities and Challenges of Web-Based and Remotely Administered Surveys for Patient Preference Studies in a Vulnerable Population.

The application of web-based and remotely administered surveys is becoming increasingly popular due to the fact that it offers numerous advantages over traditional paper-based or computer-based surveys completed in the presence of the researcher. However, it is unclear whether complex preference elicitation tasks administered online in highly vulnerable patient populations are also feasible. This commentary discusses opportunities and challenges of conducting quantitative patient preference studies in lung cancer patients using web-based modes of data collection. We refer to our recent experience in the context of the Patient Preference in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project. Among the main advantages were the possibility of reaching a wider and geographically distant population in a shorter timeframe while reducing the financial costs of testing, the greater flexibility offered and the reduced burden on the patients. Some limitations were also identified and should be the object of further research, including the potential lack of inclusiveness of the research, the lack of control over who is completing the survey, a poor comprehension of the study material, and ultimately a lower level of engagement with the study. Despite these limitations, experience from the PREFER project suggests that online quantitative methods for data collection may provide a valuable method to explore preferences in vulnerable patient populations beyond the COVID-19 pandemic.

Comparing Face-to-Face and Online Data Collection Methods in Preterm and Full-Term Children: An Exploratory Study.

The COVID-19 pandemic has transformed the landscape for children's daily lives and the landscape for developmental psychology research. Pandemic-related restrictions have also significantly disrupted the traditional face-to-face methods with which developmental scientists produce research. Over the past year, developmental scientists have published on the best practices for online data collection methods; however, existing studies do not provide empirical evidence comparing online methods to face-to-face methods. In this study, we tested feasibility of online methods by examining performance on a battery of standardized and experimental cognitive assessments in a combined sample of 4- to 5-year-old preterm and full-term children, some of whom completed the battery face-to-face, and some of whom completed the battery online. First, we asked how children's performance differs between face-to-face and online format on tasks related to verbal comprehension, fluid reasoning, visual spatial, working memory, attention and executive functioning, social perception, and numerical skills. Out of eight tasks, we did not find reliable differences on five of them. Second, we explored the role of parent involvement in children's performance in the online format. We did not find a significant effect of parent involvement on children's performance. Exploratory analyses showed that the role of format did not vary for children at risk, specifically children born preterm. Our findings contribute to the growing body of literature examining differences and similarities across various data collection methods, as well as literature surrounding online data collection for continuing developmental psychology research.

Consistency of data collected through online life history calendars.

The collection of longitudinal data is crucial in some domains such as life course studies. However, prospective studies are considerably costly, and thus retrospective data are an appealing alternative. A life history calendar is a tool specifically conceived to collect retrospective data. However, although it is designed to enhance the recall process of the respondents, the accuracy of the data collected through this approach remains unknown, particularly when data is collected online. In this study, we conducted a secondary analysis of data collected from n = 5,181 respondents through an online survey regarding their sexual health. Because we inquired about the occurrence of certain events twice during the survey, once using a life calendar and once through a traditional questionnaire, we were able to perform three types of consistency checks: (1) reporting of single events, (2) age when the events occurred and (3) correct timing between two events. The main results indicated that it is generally more difficult to remember the exact age of occurrence of an event than the event itself, that the report of related events is generally coherent, and that women are generally more accurate in their answers than men. Based on our results, it is therefore possible to identify a subset of persons whose answers are more consistent throughout the survey. This study also shows that data obtained through an online life history calendar can be of a quality similar to that obtained through a traditional online questionnaire.

Have I argued with my family this week?": What questions do those with lived experience choose to monitor their bipolar disorder?

BACKGROUND: Electronic self-report mood monitoring tools for individuals with bipolar disorder (BD) are rapidly emerging and predominately employ predefined symptom-based questions. Allowing individuals to additionally choose what they monitor in relation to their BD offers the unique opportunity to capture and gain a deeper insight into patient priorities in this context. METHODS: In addition to monitoring mood symptoms with two standardised self-rated questionnaires, 308 individuals with BD participating in the Bipolar Disorder Research Network True Colours electronic mood-monitoring tool for research chose to create and complete additional personalised questions. A content analysis approach was used to analyse the content of these questions. RESULTS: 35 categories were created based on the personalised questions with the most common being physical activity and exercise, anxiety and panic, sleep and coping/stress levels. The categories were grouped into six overarching themes 1) mental health; 2) behaviour and level of functioning; 3) physical wellbeing; 4) health behaviours; 5) active self-management; and, 6) interpersonal. LIMITATIONS: The average age of the sample was around 50 years meaning our findings may not be generalisable to younger individuals with BD. CONCLUSIONS: Aspects of BD important to patients in relation to longitudinal monitoring extend well beyond mood symptoms, highlighting the limitations of solely relying on standardised questions/mood rating scales based on symptoms primarily used for diagnosis. Additional symptoms and aspects of life not necessarily useful diagnostically for BD may be more important for individuals themselves to monitor and have more meaning in capturing their own experience of changes in BD severity.

Remote Data Collection During a Pandemic: A New Approach for Assessing and Coding Multisensory Attention Skills in Infants and Young Children.

In early 2020, in-person data collection dramatically slowed or was completely halted across the world as many labs were forced to close due to the COVID-19 pandemic. Developmental researchers who assess looking time (especially those who rely heavily on in-lab eye-tracking or live coding techniques) were forced to re-think their methods of data collection. While a variety of remote or online platforms are available for gathering behavioral data outside of the typical lab setting, few are specifically designed for collecting and processing looking time data in infants and young children. To address these challenges, our lab developed several novel approaches for continuing data collection and coding for a remotely administered audiovisual looking time protocol. First, we detail a comprehensive approach for successfully administering the Multisensory Attention Assessment Protocol (MAAP), developed by our lab to assess multisensory attention skills (MASks; duration of looking, speed of shifting/disengaging, accuracy of audiovisual matching). The MAAP is administered from a distance (remotely) by using Zoom, Gorilla Experiment Builder, an internet connection, and a home computer. This new data collection approach has the advantage that participants can be tested in their homes. We discuss challenges and successes in implementing our approach for remote testing and data collection during an ongoing longitudinal project. Second, we detail an approach for estimating gaze direction and duration collected remotely from webcam recordings using a post processing toolkit (OpenFace) and demonstrate its effectiveness and precision. However, because OpenFace derives gaze estimates without translating them to an external frame of reference (i.e., the participant's screen), we developed a machine learning (ML) approach to overcome this limitation. Thus, third, we trained a ML algorithm [(artificial neural network (ANN)] to classify gaze estimates from OpenFace with respect to areas of interest (AOI) on the participant's screen (i.e., left, right, and center). We then demonstrate reliability between this approach and traditional coding approaches (e.g., coding gaze live). The combination of OpenFace and ML will provide a method to automate the coding of looking time for data collected remotely. Finally, we outline a series of best practices for developmental researchers conducting remote data collection for looking time studies.